Adaptive, transparent frequency and voltage scaling of communication phases in MPI programs

ABSTRACT

Although users of high-performance computing are most interested in raw performance, both energy and power consumption have become critical concerns. Some microprocessors allow frequency and voltage scaling, which enables a system to reduce CPU performance and power when the CPU is not on the critical path. When properly directed, such dynamic frequency and voltage scaling can produce significant energy savings with little performance penalty.This paper presents an MPI runtime system that dynamically reduces CPU performance during communication phases in MPI programs. It dynamically identifies such phases and, without profiling or training, selects the CPU frequency in order to minimize energy-delay product. All analysis and subsequent frequency and voltage scaling is within MPI and so is entirely transparent to the application. This means that the large number of existing MPI programs, as well as new ones being developed, can use our system without modification. Results show that the average reduction in energy-delay product over the NAS benchmark suite is 10%---the average energy reduction is 12% while the average execution time increase is only 2.1%.

REFERENCES

• 1: N. D. Adiga et al. An overview of the BlueGene/L supercomputer. In Supercomputing, November 2002.
• 2: ASCI Purple Benchmark Suite. http://www.llnl.gov/asci/platiormstpurple/rfp/benchmarks/.
• 3: D. Bailey, J. Barton, T. Lasinski, and H. Simon. The NAS parallel benchmarks. RNR-91-002, NASA Ames Research Center, August 1991.
• 4: Ali Raza Butt, Chris Gniady, and Y. Charlie Hu. The performance impact of kernel prefetching on buffer cache replacement algorithms. In SIGMETRICS, pages 157--168, 2005.
• 5: Rong Ge , Xizhou Feng , Kirk W. Cameron, Performance-constrained Distributed DVS Scheduling for Scientific Applications on Power-aware Clusters, Proceedings of the 2005 ACM/IEEE conference on Supercomputing, p.34, November 12-18, 2005 [doi>10.1109/SC.2005.57]
• 6: Enrique V. Carrera , Eduardo Pinheiro , Ricardo Bianchini, Conserving disk energy in network servers, Proceedings of the 17th annual international conference on Supercomputing, June 23-26, 2003, San Francisco, CA, USA [doi>10.1145/782814.782829]
• 7: Jeffrey S. Chase , Darrell C. Anderson , Prachi N. Thakar , Amin M. Vahdat , Ronald P. Doyle, Managing energy and server resources in hosting centers, Proceedings of the eighteenth ACM symposium on Operating systems principles, October 21-24, 2001, Banff, Alberta, Canada
• 8: Guilin Chen, Konrad Malkowski, Mahmut Kandemir, and Padma Raghavan. Reducing power with performance contraints for parallel sparse applications. In Workshop on High-Performance, Power-Aware Computing, April 2005.
• 9: Ashutosh S. Dhodapkar , James E. Smith, Comparing Program Phase Detection Techniques, Proceedings of the 36th annual IEEE/ACM International Symposium on Microarchitecture, p.217, December 03-05, 2003
• 10: Elmootazbellah Elnozahy, Michael Kistler, and Ramakrishnan Rajamony. Energy conservation policies for web servers. In Usenix Symposium on Internet Technologies and Systems, 2003.
• 11: E. N. (Mootaz) Elnozahy, Michael Kistler, and Ramakrishnan Rajamony. Energy-efficient server clusters. In Workshop on Mobile Computing Systems and Applications, Feb 2002.
• 12: Mark E. Femal. Non-uniform power distribution in data centers for safely overprovisioning circuit capacity and booasting throughput. Master's thesis, North Carolina State University, Raleigh, NC, May 2005.
• 13: Vincent W. Freeh , David K. Lowenthal, Using multiple energy gears in MPI programs on a power-scalable cluster, Proceedings of the tenth ACM SIGPLAN symposium on Principles and practice of parallel programming, June 15-17, 2005, Chicago, IL, USA [doi>10.1145/1065944.1065967]
• 14: Vincent W. Freeh , Feng Pan , Nandini Kappiah , David K. Lowenthal , Rob Springer, Exploring the Energy-Time Tradeoff in MPI Programs on a Power-Scalable Cluster, Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Papers, p.4.1, April 04-08, 2005 [doi>10.1109/IPDPS.2005.214]
• 15: Chris Gniady, Ali Raza Butt, and Y. Charlie Hu. Program-counter-based pattern classification in buffer caching. In OSDI, pages 395--408, 2004.
• 16: Chris Gniady , Y Charlie Hu , Yung-Hsiang Lu, Program Counter Based Techniques for Dynamic Power Management, Proceedings of the 10th International Symposium on High Performance Computer Architecture, p.24, February 14-18, 2004 [doi>10.1109/HPCA.2004.10021]
• 17: Richard Goering. Current physical design tools come up short. EE Times, April 14 2000.
• 18: Chung-hsing Hsu , Wu-chun Feng, A Power-Aware Run-Time System for High-Performance Computing, Proceedings of the 2005 ACM/IEEE conference on Supercomputing, p.1, November 12-18, 2005 [doi>10.1109/SC.2005.3]
• 19: Chung-Hsing Hsu and Wu-chun Feng. Effective dynamic-voltage scaling through CPU-boundedness detection. In Fourth IEEE/ACM Workshop on Power-Aware Computing Systems, December 2004.
• 20: Michael C. Huang , Jose Renau , Josep Torrellas, Positional adaptation of processors: application to energy reduction, Proceedings of the 30th annual international symposium on Computer architecture, June 09-11, 2003, San Diego, California
• 21: Nandini Kappiah , Vincent W. Freeh , David K. Lowenthal, Just In Time Dynamic Voltage Scaling: Exploiting Inter-Node Slack to Save Energy in MPI Programs, Proceedings of the 2005 ACM/IEEE conference on Supercomputing, p.33, November 12-18, 2005 [doi>10.1109/SC.2005.39]
• 22: Charles Lefurgy , Karthick Rajamani , Freeman Rawson , Wes Felter , Michael Kistler , Tom W. Keller, Energy Management for Commercial Servers, Computer, v.36 n.12, p.39-48, December 2003 [doi>10.1109/MC.2003.1250880]
• 23: Athanasios E. Papathanasiou and Michael L. Scott. Energy efficiency through burstiness. In Workshop on Mobile Computing Systems and Applications, October 2003.
• 24: Eduardo Pinheiro, Ricardo Bianchini, Enrique V. Carrera, and Taliver Heath. Load balancing and unbalancing for power and performance in cluster-based systems. In Workshop on Compilers and Operating Systems for Low Power, September 2001.
• 25: Rolf Rabenseifner, Automatic Profiling of MPI Applications with Hardware Performance Counters, Proceedings of the 6th European PVM/MPI Users' Group Meeting on Recent Advances in Parallel Virtual Machine and Message Passing Interface, p.35-42, September 26-29, 1999
• 26: Vivek Sharma , Arun Thomas , Tarek Abdelzaher , Kevin Skadron , Zhijian Lu, Power-aware QoS Management in Web Servers, Proceedings of the 24th IEEE International Real-Time Systems Symposium, p.63, December 03-05, 2003
• 27: Timothy Sherwood , Erez Perelman , Greg Hamerly , Brad Calder, Automatically characterizing large scale program behavior, Proceedings of the 10th international conference on Architectural support for programming languages and operating systems, October 05-09, 2002, San Jose, California
• 28: Robert Springer , David K. Lowenthal , Barry Rountree , Vincent W. Freeh, Minimizing execution time in MPI programs on an energy-constrained, power-scalable cluster, Proceedings of the eleventh ACM SIGPLAN symposium on Principles and practice of parallel programming, March 29-31, 2006, New York, New York, USA [doi>10.1145/1122971.1123006]
• 29: Michael S. Warren , Eric H. Weigle , Wu-Chun Feng, High-density computing: a 240-processor Beowulf in one cubic meter, Proceedings of the 2002 ACM/IEEE conference on Supercomputing, p.1-11, November 16, 2002, Baltimore, Maryland
• 30: Qingbo Zhu , Francis M. David , Christo F. Devaraj , Zhenmin Li , Yuanyuan Zhou , Pei Cao, Reducing Energy Consumption of Disk Storage Using Power-Aware Cache Management, Proceedings of the 10th International Symposium on High Performance Computer Architecture, p.118, February 14-18, 2004 [doi>10.1109/HPCA.2004.10022]

INDEX TERMS

Primary Classification:
  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.3 Concurrent Programming
          Subjects: Parallel programming

Additional Classification:
  C. Computer Systems Organization
  C.1 PROCESSOR ARCHITECTURES
      C.1.2 Multiple Data Stream Architectures (Multiprocessors)
          Subjects: Multiple-instruction-stream, multiple-data-stream processors (MIMD)
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Performance attributes; Design studies

  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.4 Processors
          Subjects: Run-time environments

General Terms:
Design