Astronomical real-time streaming signal processing on a Blue Gene/L supercomputer

ABSTRACT

LOFAR is the first of a new generation of radio telescopes, that combines the signals from many thousands of simple, fixed antennas, rather than from expensive dishes. Its revolutionary design and unprecedented size enables observations in a frequency range that could hardly be observed before, and allows the study of a vast amount of new science cases.In this paper, we describe a novel approach to process realtime, streaming telescope data in software, using a supercomputer. The desire for a flexible and reconfigurable instrument demands a software solution, where traditionally customized hardware was used. This, and LOFAR's exceptional real-time, streaming signalprocessing requirements compel the use of a supercomputer. We focus on the LOFAR CEntral Processing facility (CEP), that combines the signals of all LOFAR stations. CEP consists of a 12,288-core IBM Blue Gene/L supercomputer, embedded in several conventional clusters.We describe a highly optimized implementation that will do the bulk of the central signal processing on the Blue Gene/L, namely PolyPhase Filtering, Delay Compensation, and Correlation. Measurements show that we reach exceptionally high computational performance (up to 98% of the theoretical floating-point peak performance). We also discuss how we handle external I/O performance limitations into and out of the Blue Gene/L, to obtain sufficient bandwidth for LOFAR.

REFERENCES

• 1: H.R. Butcher. LOFAR: First of a New Generation of Radio Telescopes. Proceedings of the SPIE, 5489:537--544, October 2004.
• 2: A.G. de Bruyn, R.P. Fender, J.M.E. Kuijpers, G.K. Miley, R. Ramachandran, H.J.A. Röttgering, B.W. Stappers, M.A.M. van de Weygaert, and M.P. van Haarlem. Exploring the Universe with the Low Frequency Array, A Scientific Case, September 2002. http://www.lofar.org/PDF/NL-CASE-1.0.pdf.
• 3: J. Lorenz, S. Kral, F. Franchetti, and C.W. Ueberhuber. Vectorization Techniques for the Blue Gene/L Double FPU. IBM Journal of Research and Development, 49(2/3):437--446, March 2005.
• 4: C. Phillips, T. Tzioumis, A. Deller, S. Tingay, C. Harris, and K. Haines. Electronic Transmission and Computation of Very Long Baseline Interferometry and Its Application to Next Generation Radio Telescopes. Poster in the first IEEE International Conference on e-Science and Grid Computing, December 2005.
• 5: J.J. Ritsko, I. Ames, S.I. Raider, and J.H. Robinson, editors. Blue Gene, volume 49, number 2/3 of IBM Journal of Research and Development. IBM Corporation, March/May 2005.
• 6: K. van der Schaaf, C. Broekema, G. van Diepen, and E. van Meijeren. The LOFAR Central Processing Facility Architecture. Experimental Astronomy, 17:43--58, 2005.
• 7: C. M. de Vos , K. Van der Schaaf , J. D. Bregman, Cluster Computers and Grid Processing in the First Radio-Telescope of a New Generation, Proceedings of the 1st International Symposium on Cluster Computing and the Grid, p.156, May 15-18, 2001
• 8: Private discussions with people from the Joint Institute for VLBI in Europe, March 2006.

INDEX TERMS

Primary Classification:
  J. Computer Applications
  J.2 PHYSICAL SCIENCES AND ENGINEERING
      Subjects: Astronomy

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.4 Distributed Systems
          Subjects: Distributed applications
  C.3 SPECIAL-PURPOSE AND APPLICATION-BASED SYSTEMS
      Subjects: Signal processing systems
  C.5 COMPUTER SYSTEM IMPLEMENTATION
      C.5.1 Large and Medium ("Mainframe") Computers

  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.3 Concurrent Programming

  J. Computer Applications
  J.7 COMPUTERS IN OTHER SYSTEMS
      Subjects: Real time

General Terms:
Algorithms, Measurement, Performance

Keywords:
Blue Gene/L, LOFAR, correlator, filtering