Blue matter on blue gene/L: massively parallel computation for biomolecular simulation

ABSTRACT

This paper provides an overview of the Blue Matter application development effort within the Blue Gene project that supports our scientific simulation efforts in the areas of protein folding and membrane-protein systems. The design philosophy of the Blue Gene/L architecture relies on large numbers of power efficient nodes (whose technology is derived from the world of embedded microprocessors) to enable packing of many such nodes into a small volume to achieve high performance. In order for an application to exploit the potential of this architecture, the application must scale well to large node counts. Because the scientific goals of the project entail simulating very long time-scales, up to microseconds, strong scaling of a fixed size problem to these large node counts is a requirement. In pursuit of this objective we have considered a variety of parallel decompositions and explored ways to exploit and map algorithms onto the two primary high performance interconnects provided by the Blue Gene architecture, the 3D-torus network and the collective network. Our current version of the application continues to speed up through 4096 nodes and is being used for studies of a protein/lipid system (for which some results have already been published) and for protein folding/unfolding simulations.

REFERENCES

• 1: F. Allen , G. Almasi , W. Andreoni , D. Beece , B. J. Berne , A. Bright , J. Brunheroto , C. Cascaval , J. Castanos , P. Coteus , P. Crumley , A. Curioni , M. Denneau , W. Donath , M. Eleftheriou , B. Fitch , B. Fleischer , C. J. Georgiou , R. Germain , M. Giampapa , D. Gresh , M. Gupta , R. Haring , H. Ho , P. Hochschild , S. Hummel , T. Jonas , D. Lieber , G. Martyna , K. Maturu , J. Moreira , D. Newns , M. Newton , R. Philhower , T. Picunko , J. Pitera , M. Pitman , R. Rand , A. Royyuru , V. Salapura , A. Sanomiya , R. Shah , Y. Sham , S. Singh , M. Snir , F. Suits , R. Swetz , W. C. Swope , N. Vishnumurthy , T. J. C. Ward , H. Warren , R. Zhou, Blue Gene: a vision for protein science using a petaflop supercomputer, IBM Systems Journal, v.40 n.2, p.310-327, February 2001
• 2: M. P. Allen , D. J. Tildesley, Computer simulation of liquids, Clarendon Press, New York, NY, 1989
• 3: G. Almasi et al. Design and implementation of message-passing services for the Blue Gene/L supercomputer. IBM Journal of Research and Development, 49(2/3):393--406, 2005. http://www.research.ibm.com/journal/rd/492/almasi.pdf.
• 4: J. Bader and D. Chandler. Computer simulation study of the mean forces between ferrous and ferric ions in water. The Journal of Physical Chemistry, 96(15), 1992.
• 5: S. Chatterjee et al. Design and exploitation of a high-performance SIMD floating-point unit for Blue Gene/L. IBM Journal of Research and Development, 49(2/3):377--392, 2005. http://www.research.ibm.com/journal/rd/492/chatterjee.pdf.
• 6: M. Deserno and C. Holm. How to mesh up ewald sums. i. a theoretical and numerical comparison of various particle mesh routines. J. Chem. Phys., 109(18):7678--7693, 1998.
• 7: M. Eleftheriou, B. Fitch, A. Rayshubskiy, T. Ward, and R. Germain. Scalable framework for 3d FFTs on the Blue Gene/L supercomputer: Implementation and early performance measurements. IBM Journal of Research and Development, 49(2/3):457--464, 2005. http://www.research.ibm.com/journal/rd/492/eleftheriou.pdf.
• 8: R. Enenkel et al. Custom math functions for molecular dynamics. IBM Journal of Research and Development, 49(2/3):465--474, 2005. http://www.research.ibm.com/journal/rd/492/enenkel.pdf.
• 9: B. G. Fitch , R. S. Germain , M. Mendell , J. Pitera , M. Pitman , A. Rayshubskiy , Y. Sham , F. Suits , W. Swope , T. J. C. Ward , Y. Zhestkov , R. Zhou, Blue Matter, an application framework for molecular simulation on blue gene, Journal of Parallel and Distributed Computing, v.63 n.7-8, p.759-773, July 2003 [doi>10.1016/S0743-7315(03)00084-4]
• 10: A. Gara et al. Overview of the Blue Gene/L system architecture. IBM Journal of Research and Development, 49(2/3):195--212, 2005. http://www.research.ibm.com/journal/rd/492/gara.pdf.
• 11: M. Giampapa et al. Blue Gene/L advanced diagnostics environment. IBM Journal of Research and Development, 49(2/3):319--332, 2005. http://www.research.ibm.com/journal/rd/492/giampapa.pdf.
• 12: M. Karplus and J. McCammon. Molecular dynamics simulations of biomolecules. Nature Structural Biology, 9(9):646--652, September 2002.
• 13: J. Lorenz, S. Kral, F. Franchetti, and C. Ueberhuber. Vectorization techniques for the Blue Gene/L double FPU. IBM Journal of Research and Development, 49(2/3):437--446, 2005. http://www.research.ibm.com/journal/rd/492/lorenz.pdf.
• 14: J. Moreira et al. Blue Gene/L programming and operating environment. IBM Journal of Research and Development, 49(2/3):367--376, 2005. http://www.research.ibm.com/journal/rd/492/moreira.pdf.
• 15: Lars Nyland , Jan Prins , Ru Huai Yun , Jan Hermans , Hye-Chung Kum , Lei Wang, Achieving scalable parallel molecular dynamics using dynamic spatial domain decomposition techniques, Journal of Parallel and Distributed Computing, v.47 n.2, p.125-138, Dec. 15, 1997 [doi>10.1006/jpdc.1997.1408]
• 16: M. C. Pitman, A. Grossfield, F. Suits, and S. E. Feller. Role of cholesterol and polyunsaturated chains in lipid-protein interactions: Molecular dynamics simulation of rhodopsin in a realistic membrane environment. Journal of the American Chemical Society, 127(13):4576--4577, 2005. http://dx.doi.org/10.1021/ja042715y.
• 17: S. Plimpton and B. Hendrickson. A new parallel method for molecular dynamics simulation of macromolecular systems. Journal of Computational Chemistry, 17(3):326--337, 1996.
• 18: C. Wait. IBM PowerPC 440 FPU with complex-arithmetic extensions. IBM Journal of Research and Development, 49(2/3):249--254, 2005. http://www.research.ibm.com/journal/rd/492/wait.pdf.

INDEX TERMS

Primary Classification:
  J. Computer Applications
  J.3 LIFE AND MEDICAL SCIENCES
      Subjects: Biology and genetics

Additional Classification:
  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.3 Concurrent Programming
          Subjects: Parallel programming

General Terms:
Algorithms, Performance

Keywords:
biochip, microfluidics, physical design, synthesis