John W. Romein
ABSTRACT
LOFAR is the first of a new generation of radio telescopes.Rather than using expensive dishes, it forms a distributed sensor network that combines the signals from many thousands of simple antennas. Its revolutionary design allows observations in a frequency range that has hardly been studied before.
Another novel feature of LOFAR is the elaborate use of software to process data, where traditional telescopes use customized hardware. This dramatically increases flexibility and substantially reduces costs, but the high processing and bandwidth requirements compel the use of a supercomputer. The antenna signals are centrally combined, filtered, optionally beam-formed, and correlated by an IBM Blue Gene/P.
This paper describes the implementation of the so-called correlator. To meet the real-time requirements, the application is highly optimized, and reaches exceptionally high computational and I/O efficiencies. Additionally, we study the scalability of the system, and show that it scales well beyond the requirements. The optimizations allows us to use only half the planned amount of resources, and process 50% more telescope data, significantly improving the effectiveness of the entire telescope.
- H.R. Butcher. LOFAR: First of a New Generation of Radio Telescopes. Proceedings of the SPIE, 5489:537--544, October 2004.Google Scholar
Cross Ref
- A.G. de Bruyn et al. Exploring the Universe with the Low Frequency Array, A Scientific Case, September 2002. http://www.lofar.org/PDF/NL-CASE-1.0.pdf.Google Scholar
- A. Deller, S. Tingay, M. Bailes, and C. West. DiFX: A Software Correlator for Very Long Baseline Interferometry Using Multiprocessor Computing Environments. Astronomical Society of the Pacific, 119:318--336, 2007.Google ScholarCross Ref
- J. Hessels, B. Stappers, and J. van Leeuwen. The Radio Sky on Short Timescales with LOFAR: Pulsars and Fast Transients. In The Low-Frequency Radio Universe, ASP Conference Series. To appear. http://arxiv.org/pdf/0903.1447.Google Scholar
- K. Iskra, J.W. Romein, K. Yoshii, and P. Beckman. ZOID: I/OForwarding Infrastructure for Petascale Architectures. In ACM SIGPLAN Symposium on Principles and Practice on Parallel Programming (PPoPP'08), pages 153--162, Salt Lake City, UT, February 2008. Google ScholarDigital Library
- N. Kruithof and D. Marchal. Real-time Software Correlation. In International Workshop on Distributed Cooperative Laboratories (INGRID' 08), April 2008. http://www.jive.nl/dokuwiki/doku.php/scarie:scarie.Google Scholar
- J. Lorenz, S. Kral, F. Franchetti, and C.W. Ueberhuber. Vectorization Techniques for the Blue Gene/L Double FPU. IBM Journal of Research and Development, 49(2/3):437--446, March 2005. Google ScholarDigital Library
- R.V. van Nieuwpoort and J.W. Romein. Using Many-Core Hardware to Correlate Radio Astronomy Signals. In ACM International Conference on Supercomputing (ICS'09), pages 440--449, New York, NY, June 2009. Google ScholarDigital Library
- S. Ord, L. Greenhill, R. Wayth, D. Mitchell, K. Dale, H. Pfister, and G. Edgar. GPUs for data processing in the MWA. In Astronomical Data Analysis Software and Systems (ADASS XVIII), November 2008. To appear. http://arxiv.org/abs/0902.0915.Google Scholar
- J.W. Romein. FCNP: Fast I/O on the Blue Gene/P. In Parallel and Distributed Processing Techniques and Applications (PDPTA'09), Las Vegas, NV, July 2009.Google Scholar
- J.W. Romein, P.C. Broekema, E. van Meijeren, K. van der Schaaf, and W.H. Zwart. Astronomical Real-Time Streaming Signal Processing on a Blue Gene/L Supercomputer. In ACM Symposium on Parallel Algorithms and Architectures (SPAA'06), pages 59--66, Cambridge, MA, July 2006. Google ScholarDigital Library
- IBM Blue Gene team. Overview of the IBM Blue Gene/P Project. IBM Journal of Research and Development, 52(1/2), January/March 2008. Google ScholarDigital Library
- M. de Vos, A.W. Gunst, and R. Nijboer. The LOFAR Telescope: System Architecture and Signal Processing. Proceedings of the IEEE. To appear.Google Scholar
- K. Yoshii, K. Iskra, H. Naik, and P.C. Broekema P. Beckman. Performance and Scalability Evaluation of "Big Memory" on Blue Gene Linux. International Journal of High Performance Computing. To appear. Google ScholarDigital Library
Index Terms
- The LOFAR correlator: implementation and performance analysis
Recommendations
The LOFAR correlator: implementation and performance analysis
PPoPP '10LOFAR is the first of a new generation of radio telescopes.Rather than using expensive dishes, it forms a distributed sensor network that combines the signals from many thousands of simple antennas. Its revolutionary design allows observations in a ...
Astronomical real-time streaming signal processing on a Blue Gene/L supercomputer
SPAA '06: Proceedings of the eighteenth annual ACM symposium on Parallelism in algorithms and architecturesLOFAR is the first of a new generation of radio telescopes, that combines the signals from many thousands of simple, fixed antennas, rather than from expensive dishes. Its revolutionary design and unprecedented size enables observations in a frequency ...
Using many-core hardware to correlate radio astronomy signals
ICS '09: Proceedings of the 23rd international conference on SupercomputingA recent development in radio astronomy is to replace traditional dishes with many small antennas. The signals are combined to form one large, virtual telescope. The enormous data streams are cross-correlated to filter out noise. This is especially ...
Reviews
Joseph M. Arul
Instead of using traditional telescopes to monitor the sky and combine all the signals centrally, to generate sky images, this research considers and presents the low-frequency array (LOFAR) radio telescope. This groundbreaking research observes many directions and switches instantaneously, in any direction over a range of 1,300 kilometers (km), linking three different continents at a time. The system uses software to process the data in real time, rather than going the traditional route and using field-programmable gate arrays (FPGAs). Since this system uses a Blue Gene/L supercomputer to process a huge volume of data in a real-time environment, the study focuses on performance aspects, real-time behavior, and the correlator's scalability characteristics. This particular application uses three different types of nodes: input/output (I/O) nodes, compute nodes, and storage nodes. The authors developed a fast collective-network protocol (FCNP) library to communicate between the I/O nodes and the compute nodes. The compute nodes perform various steps, such as polyphase filtering, fast Fourier transforms (FFTs), and bandpass filtering. Since a compute node can consume a lot of time doing computation, a software pipelining technique could be considered and implemented to improve its efficiency. As Romein et al. note, it is important to closely tie together the I/O nodes and compute nodes, in order to constantly feed data to the compute nodes. Since this application involves processing a huge volume of data, software pipelining techniques could greatly improve data processing performance-in the conclusion, the authors mention this as a future goal of the project. For interested researchers, there are still many areas where the efficient implementation of various steps could improve the overall performance of the application. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments