Evaluating the effect of replacing CNK with linux on the compute-nodes of blue gene/l

ABSTRACT

The Blue Gene machines in production today run a small single-user, single-process kernel (CNK) having a limited functionality. Motivated by the desire to provide applications with a much richer operating environment, we evaluate the effect of replacing CNK with a standard Linux kernel on the compute nodes of Blue Gene/L. We show that with a relatively small amount of effort we were able to improve benchmark performance under Linux up to a level that is comparable to CNK.

REFERENCES

• 1: The MPICH and MPICH2 homepage. http://www-unix.mcs.anl.gov/mpi/mpich.
• 2: NR Adiga , G Almasi , GS Almasi , Y Aridor , R Barik , D Beece , R Bellofatto , G Bhanot , R Bickford , M Blumrich , AA Bright , J Brunheroto , C Caşcaval , J Castaños , W Chan , L Ceze , P Coteus , S Chatterjee , D Chen , G Chiu , TM Cipolla , P Crumley , KM Desai , A Deutsch , T Domany , MB Dombrowa , W Donath , M Eleftheriou , C Erway , J Esch , B Fitch , J Gagliano , A Gara , R Garg , R Germain , ME Giampapa , B Gopalsamy , J Gunnels , M Gupta , F Gustavson , S Hall , RA Haring , D Heidel , P Heidelberger , LM Herger , D Hoenicke , RD Jackson , T Jamal-Eddine , GV Kopcsay , E Krevat , MP Kurhekar , AP Lanzetta , D Lieber , LK Liu , M Lu , M Mendell , A Misra , Y Moatti , L Mok , JE Moreira , BJ Nathanson , M Newton , M Ohmacht , A Oliner , V Pandit , RB Pudota , R Rand , R Regan , B Rubin , A Ruehli , S Rus , RK Sahoo , A Sanomiya , E Schenfeld , M Sharma , E Shmueli , S Singh , P Song , V Srinivasan , BD Steinmacher-Burow , K Strauss , C Surovic , R Swetz , T Takken , RB Tremaine , M Tsao , AR Umamaheshwaran , P Verma , P Vranas , TJC Ward , M Wazlowski , W Barrett , C Engel , B Drehmel , B Hilgart , D Hill , F Kasemkhani , D Krolak , CT Li , T Liebsch , J Marcella , A Muff , A Okomo , M Rouse , A Schram , M Tubbs , G Ulsh , C Wait , J Wittrup , M Bae , K Dockser , L Kissel , MK Seager , JS Vetter , K Yates, An overview of the BlueGene/L Supercomputer, Proceedings of the 2002 ACM/IEEE conference on Supercomputing, p.1-22, November 16, 2002, Baltimore, Maryland
• 3: G. Alm--asi, R. Bellofatto, J. Brunheroto, C. Cascaval, J. G. Castanos, L. Ceze, P. Crumley, C. Erway, J. Gagliano, D. Lieber, X. Martorell, J. E. Moreira, A. Sanomiya, and K. Strauss. An overview of the BlueGene/L system software organization. In Proceedings of Euro-Par 2003 Conference, Lecture Notes in Computer Science, Klagenfurt, Austria, August 2003. Springer-Verlag.
• 4: G. Alm--asi et al. The Design and Implementation of Message Passing Services for the BlueGene/L Supercomputer. IBM System Journal, 2004.
• 5: Astron project homepage. http://astron.nl/.
• 6: P. Beckman, K. Iskra, K. Yoshii, and S. Coghlan. The influence of operating systems on the performance of collective operations at extreme scale. In Proceedings of the 8th IEEE International Conference on Cluster Computing, Barcelona, Spain, Sept. 2006.
• 7: Pete Beckman , Kamil Iskra , Kazutomo Yoshii , Susan Coghlan, Operating system issues for petascale systems, ACM SIGOPS Operating Systems Review, v.40 n.2, April 2006 [doi>10.1145/1131322.1131332]
• 8: Blue brain project homepage. http://bluebrain.epfl.ch/.
• 9: Ron Brightwell , Lee Ann Fisk , David S. Greenberg , Tramm Hudson , Mike Levenhagen , Arthur B. MacCabe , Rolf Riesen, Massively parallel computing using commodity components, Parallel Computing, v.26 n.2-3, p.243-266, Feb. 2000 [doi>10.1016/S0167-8191(99)00104-0]
• 10: Ron Brightwell , Arthur B. MacCabe , Rolf Riesen, On the Appropriateness of Commodity Operating Systems for Large-Scale, Balanced Computing Systems, Proceedings of the 17th International Symposium on Parallel and Distributed Processing, p.68.1, April 22-26, 2003
• 11: R. Brightwell, R. Riesen, K. D. Underwood, T. Hudson, P. G. Bridges, and A. B. Maccabe. A performance comparison of linux and a lightweight kernel. In CLUSTER, pages 251--258, 2003.
• 12: Sayantan Chakravorty , Celso L. Mendes , Laxmikant V. Kalé , Terry Jones , Andrew Tauferner , Todd Inglett , José Moreira, HPC-Colony: services and interfaces for very large systems, ACM SIGOPS Operating Systems Review, v.40 n.2, April 2006 [doi>10.1145/1131322.1131334]
• 13: J. Dongarra, H.-W. Meuer, and E. Strohmaier. TOP500 Supercomputer Sites. Available in Web page at: http://www.netlib.org/benchmark/top500.html.
• 14: Francois Gygi , Erik W. Draeger , Martin Schulz , Bronis R. de Supinski , John A. Gunnels , Vernon Austel , James C. Sexton , Franz Franchetti , Stefan Kral , Christoph W. Ueberhuber , Juergen Lorenz, Large-scale electronic structure calculations of high-Z metals on the BlueGene/L platform, Proceedings of the 2006 ACM/IEEE conference on Supercomputing, November 11-17, 2006, Tampa, Florida [doi>10.1145/1188455.1188502]
• 15: FUSE Homepage. http://fuse.sourceforge.net/.
• 16: Timothy G. Mattson , David Scott , Stephen R. Wheat, A TeraFLOP Supercomputer in 1996: The ASCI TFLOP System, Proceedings of the 10th International Parallel Processing Symposium, p.84-93, April 15-19, 1996
• 17: Juan Navarro , Sitararn Iyer , Peter Druschel , Alan Cox, Practical, transparent operating system support for superpages, ACM SIGOPS Operating Systems Review, v.36 n.SI, Winter 2002 [doi>10.1145/844128.844138]
• 18: NAS Parallel Benchmarks. http://www.nas.nasa.gov/Software/NPB.
• 19: Fabrizio Petrini , Darren J. Kerbyson , Scott Pakin, The Case of the Missing Supercomputer Performance: Achieving Optimal Performance on the 8,192 Processors of ASCI Q, Proceedings of the 2003 ACM/IEEE conference on Supercomputing, p.55, November 15-21, 2003

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Design studies

Additional Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.0 General

General Terms:
Experimentation, Performance

Keywords:
CNK, NPB, bluegene, kernel, linux