The role of MPI in development time: a case study

ABSTRACT

There is widespread belief in the computer science community that MPI is a difficult and time-intensive approach to developing parallel software. Nevertheless, MPI remains the dominant programming model for HPC systems, and many projects have made effective use of it. It remains unknown how much impact the use of MPI truly has on the productivity of computational scientists.

In this paper, we examine a mature, ongoing HPC project, the Flash Center at the University of Chicago, to understand how MPI is used and to estimate the time that programmers spend on MPI-related issues during development. Our analysis is based on an examination of the source code, version control history, and regression testing history of the software. Based on our study, we estimate that about 20% of the development effort is related to MPI. This implies a maximum productivity improvement of 25% for switching to an alternate parallel programming model.

REFERENCES

• 1: Gene M. Amdahl, Validity of the single processor approach to achieving large scale computing capabilities, Proceedings of the April 18-20, 1967, spring joint computer conference, April 18-20, 1967, Atlantic City, New Jersey [doi>10.1145/1465482.1465560]
• 2: Victor R. Basili , Jeffrey C. Carver , Daniela Cruzes , Lorin M. Hochstein , Jeffrey K. Hollingsworth , Forrest Shull , Marvin V. Zelkowitz, Understanding the High-Performance-Computing Community: A Software Engineer's Perspective, IEEE Software, v.25 n.4, p.29-36, July 2008 [doi>10.1109/MS.2008.103]
• 3: M. R. Beinoff and E. D. Lazowska, "Computational science: Ensuring America's competitiveness / President's Information Technology Advisory Committee (PITAC)," National Coordination Office for Information Technology Research & Development, Arlington, VA 2005.
• 4: Jeffrey C. Carver , Richard P. Kendall , Susan E. Squires , Douglass E. Post, Software Development Environments for Scientific and Engineering Software: A Series of Case Studies, Proceedings of the 29th international conference on Software Engineering, p.550-559, May 20-26, 2007 [doi>10.1109/ICSE.2007.77]
• 5: B.L. Chamberlain , D. Callahan , H.P. Zima, Parallel Programmability and the Chapel Language, International Journal of High Performance Computing Applications, v.21 n.3, p.291-312, August 2007 [doi>10.1177/1094342007078442]
• 6: R. Choy and A. Edelman, "MATLAB*P 2.0: A unified parallel MATLAB", MIT DSpace, Computer Science collection, January 2003.
• 7: Philippe Charles , Christian Grothoff , Vijay Saraswat , Christopher Donawa , Allan Kielstra , Kemal Ebcioglu , Christoph von Praun , Vivek Sarkar, X10: an object-oriented approach to non-uniform cluster computing, Proceedings of the 20th annual ACM SIGPLAN conference on Object oriented programming, systems, languages, and applications, October 16-20, 2005, San Diego, CA, USA
• 8: W. W. Carlson, D. E. Culler, K. A. Yellick, E. Brooks and K. Warren, "Introduction to UPC and language specification", Center for Computing Sciences Technical Report, CCS-TR-99-157, May 1999.
• 9: W. S. Cleveland, "Graphical methods for data presentation: Full scale breaks, dot charts, and multibased logging," The American Statistician, Vol. 38, November 1984.
• 10: J. Dongarra, R. Graybill, W. Harrod, R. Lucas, E. Lusk, P. Luszczek, J. McMahon, A. Snavely, J. Vetter, K. Yelick, S. Alam, R. Campbell, L. Carrington, T.-Y. Chen, O. Khalili, J. Meredith, and M. Tikir," DARPA's HPCS program: History, models, tools, languages," in Advances in Computers, Vol. 72, Elsevier, June 2008.
• 11: H. C. Edwards and J. R. Stewart. "Sierra, a software environment for developing complex multiphysics applications". In K. J. Bathe, editor, Computational Fluid and Solid Mechanics. Proc. First MIT Conf., pp 1147--1150, Elsevier, 2001.
• 12: Stephen G. Eick , Todd L. Graves , Alan F. Karr , J. S. Marron , Audris Mockus, Does Code Decay? Assessing the Evidence from Change Management Data, IEEE Transactions on Software Engineering, v.27 n.1, p.1-12, January 2001 [doi>10.1109/32.895984]
• 13: B. Fryxell, K. Olson, P. Ricker, F. X. Timmes, M. Zingale, D. Q. Lamb, P. MacNeice, R. Rosner, J. W. Truran, and H. Tufo, "FLASH: An Adaptive Mesh Hydrodynamics Code for Modeling Astrophysical Thermonuclear Flashes", The Astrophysical Journal Supplement Series, Vol. 131, Iss. 1, pp. 273--334, Nov 2000.
• 14: S. L. Graham, M. Snir, and C. A. Patterson, "Getting up to speed: The future of supercomputing," National Academies Press, 2004.
• 15: T. Goodale, G. Allen, G. Lanfermann, J. Massó, T. Radke, E. Seidel, and J. Shalf. "The Cactus framework and toolkit: design and applications," Vector and Parallel Processing - VECPAR 2002, 5th International Conference. Springer, 2003.
• 16: Federal plan for high-end computing: Report of the high-end computing revitalization task force (HECRTF), May 10, 2004.
• 17: Lorin Hochstein , Victor R. Basili, The ASC-Alliance Projects: A Case Study of Large-Scale Parallel Scientific Code Development, Computer, v.41 n.3, p.50-58, March 2008 [doi>10.1109/MC.2008.101]
• 18: L. Hochstein, J. Carver, F. Shull, S. Asgari, V. R. Basili, J. Hollingsworth, and M. Zelkowitz, "HPC programmer productivity: A case study of novice HPC programmers," ACM/IEEE Conference on Supercomputing (SC '05), 2005.
• 19: L. Hochstein, T. Nakamura, V. R. Basili, S. Asgari, M. V. Zelkowitz, Jeffrey K. Hollingsworth, Forrest Shull, Jeffrey Carver, Martin Voelp, Nico Zazworka, and Philip Johnson, "Experiments to Understand HPC Time to Development", CTWatch Quarterly, Vol 2, No. 4A, November 2006.
• 20: Lorin Hochstein , Victor R. Basili , Uzi Vishkin , John Gilbert, A pilot study to compare programming effort for two parallel programming models, Journal of Systems and Software, v.81 n.11, p.1920-1930, November, 2008 [doi>10.1016/j.jss.2007.12.798]
• 21: L. V. Kale and S. Krishnan, "Charm++: parallel programming with message-driven objects," G. V. Wilson, P. Lu (Eds.), Parallel Programming Using C++; MIT Press, Cambridge, MA, pp. 175--213, 1996.
• 22: G. Kumfert, and T. Epperly, "Software in the DOE: The Hidden Overhead of 'The Build'", Lawrence Livermore National Laboratory Technical Report, UCRL-ID-147343, Feb 28, 2002.
• 23: P. MacNeice, K. M. Olson, C. Mobarry, R. deFainchtein, and C. Packer, "PARAMESH: A parallel adaptive mesh refinement community toolkit.", Computer Physics Communications, Vol. 126, pp. 330--354, 2000.
• 24: Message Passing Interface Forum. MPI: A Message Passing Interface Standard, Version 1.1, June 1995.
• 25: K. Olson and P. MacNeice, "An Over of the PARAMESH AMR Software and Some of Its Applications", in Adaptive Mesh Refinement-Theory and Applications, Proceedings of the Chicago Workshop on Adaptive Mesh Refinement Methods, Series: Lecture Notes in Computational Science and Engineering, vol. 41, eds. T. Plewa, T. Linde, and G. Weirs, Springer, Berlin, 2005.
• 26: K. Olson, "PARAMESH: A Parallel Adaptive Grid Tool", in Parallel Computational Fluid Dynamics 2005: Theory and Applications: Proceedings of the Parallel CFD Conference, College Park, MD, U.S.A., eds. A. Deane, A. Ecer, G. Brenner, D. Emerson, J. McDonough, J. Periaux, N. Satofuka, and D. Tromeur-Dervout, Elsevier, 2006.
• 27: D. E. Perry, N. A. Staudenmayer, and L. G. Votta, "Understanding and improving time usage in software development," in Software Process. Vol. 4, A. Fuggetta and A. Wolf, Eds.: John Wiley and Sons, 1996.
• 28: C Pilato , Ben Collins-Sussman , Brian Fitzpatrick, Version Control with Subversion, O'Reilly Media, Inc., 2008
• 29: Roger S. Pressman, Software Engineering: A Practitioner's Approach, McGraw-Hill Higher Education, 2001
• 30: J. V. W. Reynders, P. J. Hinker, J. C. Cummings, S. R. Atlas, S. Banerjee, W. F. Humphrey, S. R. Karmesin, K. Keahy, M. Srikant, and M. Tholburn, "POOMA: A framework for scientific simulation on parallel architectures", in Parallel Programming for C++, pp. 547--558, MIT Press, 1996.
• 31: Ben Shneiderman, Tree visualization with tree-maps: 2-d space-filling approach, ACM Transactions on Graphics (TOG), v.11 n.1, p.92-99, Jan. 1992 [doi>10.1145/102377.115768]
• 32: R. K. Yin, Case study research: Design and methods, Third ed.: Sage Publications, 2002.

INDEX TERMS

Primary Classification:
  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.3 Concurrent Programming

Additional Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS

  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.0 General
      D.2.8 Metrics

General Terms:
Design, Experimentation, Measurement, Performance

Keywords:
MPI, case study, debugging, effort, productivity