OMPI: optimizing MPI programs using partial evaluation

ABSTRACT

MPI is gaining acceptance as a standard for message-passing in high-performance computing, due to its powerful and flexible support of various communication styles. However, the complexity of its API poses significant software overhead, and as a result, applicability of MPI has been restricted to rather regular, coarse-grained computations. Our OMPI (Optimizing MPI) system removes much of the excess overhead by employing partial evaluation techniques, which exploit static information of MPI calls. Because partial evaluation alone is insufficient, we also utilize template functions for further optimization. To validate the effectiveness for our OMPI system, we performed baseline as well as more extensive benchmarks on a set of application cores with different communication characteristics, on the 64-node Fujitsu AP1000 MPP. Benchmarks show that OMPI improves execution efficiency by as much as factor of two for communication-intensive application core with minimal code increase. It also performs significantly better than previous dynamic optimization technique.

REFERENCES

• 1: H. Franke, P. Hochschild, P. Pattnaik, J. Prost, and M. Snir. MPI on IBM SP1/SP2: Current status and future directions. In Proceedings of 1994 Scalable Parallel Libraries. October 1994.
• 2: K. Konishi, Y. Takano, and A. Konagaya. MPI/DE: and MPI library for Cenju-3. In MPI Developers Conference, University of Notre Dame, June 1995.
• 3: D. Sitsky and K. Hayashi. Implementing MPI for the Fujitsu AP1000/AP1000+ using Polling, Interrupts and Remote Copying. In Proceedings of Joint Symposium on Parallel Processing '96, University of Waseda, Japan. June 1996. (to be submitted)
• 4: Toshiyuki Shimizu , Takeshi Horie , Hiroaki Ishihata, Low-latency message communication support for the AP1000, Proceedings of the 19th annual international symposium on Computer architecture, p.288-297, May 19-21, 1992, Queensland, Australia
• 5: R. Wilson, R. French, C. Wilson, S. Amrasinghe, J. Anderson, S. Tjiang, S-W. Liao, C-W. Tseng, M. Hall, M. Lam, and J. Hennessy. The SUIF Compiler System. Computer Systems Laboratory, Stanford University, 1994.
• 6: Message-Passing Interface Forum. MPI: A message passing interface standard, version 1.1. June 1995.
• 7: William Gropp , Ewing Lusk , Anthony Skjellum, Using MPI: portable parallel programming with the message-passing interface, MIT Press, Cambridge, MA, 1994
• 8: Kenjiro Taura , Satoshi Matsuoka , Akinori Yonezawa, An efficient implementation scheme of concurrent object-oriented languages on stock multicomputers, Proceedings of the fourth ACM SIGPLAN symposium on Principles and practice of parallel programming, p.218-228, May 19-22, 1993, San Diego, California, United States
• 9: Thorsten von Eicken , David E. Culler , Seth Copen Goldstein , Klaus Erik Schauser, Active messages: a mechanism for integrated communication and computation, Proceedings of the 19th annual international symposium on Computer architecture, p.256-266, May 19-21, 1992, Queensland, Australia
• 10: A. Dave, Communication compilation for unreliable networks, Proceedings of the 16th International Conference on Distributed Computing Systems (ICDCS '96), p.188, May 27-30, 1996

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS

Additional Classification:
  D. Software
  D.3 PROGRAMMING LANGUAGES

  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS

General Terms:
Design, Languages, Performance, Standardization, Theory

Keywords:
MPI, message passing, partial evaluation, SUIF, communication optimization, parallel computing