Adagio: making DVS practical for complex HPC applications

ABSTRACT

Power and energy are first-order design constraints in high performance computing. Current research using dynamic voltage scaling (DVS) relies on trading increased execution time for energy savings, which is unacceptable for most high performance computing applications. We present Adagio, a novel runtime system that makes DVS practical for complex, real-world scientific applications by incurring only negligible delay while achieving significant energy savings. Adagio improves and extends previous state-of-the-art algorithms by combining the lessons learned from static energy-reducing CPU scheduling with a novel runtime mechanism for slack prediction. We present results using Adagio for two real-world programs, UMT2K and ParaDiS, along with the NAS Parallel Benchmark suite. While requiring no modification to the application source code, Adagio provides total system energy savings of 8% and 20% for UMT2K and ParaDiS, respectively, with less than 1% increase in execution time.

REFERENCES

• 1: Vasily Bulatov , Wei Cai , Jeff Fier , Masato Hiratani , Gregg Hommes , Tim Pierce , Meijie Tang , Moono Rhee , Kim Yates , Tom Arsenlis, Scalable Line Dynamics in ParaDiS, Proceedings of the 2004 ACM/IEEE conference on Supercomputing, p.19, November 06-12, 2004 [doi>10.1109/SC.2004.53]
• 2: Rong Ge , Xizhou Feng , Kirk W. Cameron, Performance-constrained Distributed DVS Scheduling for Scientific Applications on Power-aware Clusters, Proceedings of the 2005 ACM/IEEE conference on Supercomputing, p.34, November 12-18, 2005 [doi>10.1109/SC.2005.57]
• 3: Linux kernel cpufreq subsystem.
• 4: Matthew Curtis-Maury , James Dzierwa , Christos D. Antonopoulos , Dimitrios S. Nikolopoulos, Online power-performance adaptation of multithreaded programs using hardware event-based prediction, Proceedings of the 20th annual international conference on Supercomputing, June 28-July 01, 2006, Cairns, Queensland, Australia [doi>10.1145/1183401.1183426]
• 5: Yang Ding, Mahmut Kandemir, Padma Raghavan, and Mary Jane Irwin. A helper thread based EDP reduction scheme for adapting application execution in CMPs. In International Parallel and Distributed Processing Symposium, April 2008.
• 6: Vincent W. Freeh , Feng Pan , Nandini Kappiah , David K. Lowenthal , Rob Springer, Exploring the Energy-Time Tradeoff in MPI Programs on a Power-Scalable Cluster, Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Papers, p.4.1, April 04-08, 2005 [doi>10.1109/IPDPS.2005.214]
• 7: Rong Ge , Xizhou Feng , Wu-chun Feng , Kirk W. Cameron, CPU MISER: A Performance-Directed, Run-Time System for Power-Aware Clusters, Proceedings of the 2007 International Conference on Parallel Processing, p.18, September 10-14, 2007 [doi>10.1109/ICPP.2007.29]
• 8: Chung-hsing Hsu , Wu-chun Feng, A Power-Aware Run-Time System for High-Performance Computing, Proceedings of the 2005 ACM/IEEE conference on Supercomputing, p.1, November 12-18, 2005 [doi>10.1109/SC.2005.3]
• 9: Tohru Ishihara , Hiroto Yasuura, Voltage scheduling problem for dynamically variable voltage processors, Proceedings of the 1998 international symposium on Low power electronics and design, p.197-202, August 10-12, 1998, Monterey, California, United States [doi>10.1145/280756.280894]
• 10: Vincent W. Freeh , Nandini Kappiah , David K. Lowenthal , Tyler K. Bletsch, Just-in-time dynamic voltage scaling: Exploiting inter-node slack to save energy in MPI programs, Journal of Parallel and Distributed Computing, v.68 n.9, p.1175-1185, September, 2008 [doi>10.1016/j.jpdc.2008.04.007]
• 11: Lawrence Livermore National Laboratory. The ASCI Purple Benchmarks. http://www.llnl.gov/asci/platforms/purple/rfp/benchmarks, 2001.
• 12: Lawrence Livermore National Laboratory. The UMT Benchmark Code. http://www.llnl.gov/asci/platforms/purple/rfp/benchmarks/limited/umt/, January 2002.
• 13: Jian Li and Jose Martinez. Dynamic power-performance adaptation of parallel computation on chip multiprocessors. In International Symposium on High-Performance Computer Architecture, 2006.
• 14: Jian Li , Jose F. Martinez , Michael C. Huang, The Thrifty Barrier: Energy-Aware Synchronization in Shared-Memory Multiprocessors, Proceedings of the 10th International Symposium on High Performance Computer Architecture, p.14, February 14-18, 2004 [doi>10.1109/HPCA.2004.10018]
• 15: Min Yeol Lim , Vincent W. Freeh , David K. Lowenthal, Adaptive, transparent frequency and voltage scaling of communication phases in MPI programs, Proceedings of the 2006 ACM/IEEE conference on Supercomputing, November 11-17, 2006, Tampa, Florida [doi>10.1145/1188455.1188567]
• 16: Chun Liu , Anand Sivasubramaniam , Mahmut Kandemir , Mary Jane Irwin, Exploiting Barriers to Optimize Power Consumption of CMPs, Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Papers, p.5.1, April 04-08, 2005 [doi>10.1109/IPDPS.2005.211]
• 17: Bren Mochocki , Xiaobo Sharon Hu , Gang Quan, A realistic variable voltage scheduling model for real-time applications, Proceedings of the 2002 IEEE/ACM international conference on Computer-aided design, p.726-731, November 10-14, 2002, San Jose, California [doi>10.1145/774572.774679]
• 18: Bren Mochocki , Xiaobo Sharon Hu , Gang Quan, Practical On-line DVS Scheduling for Fixed-Priority Real-Time Systems, Proceedings of the 11th IEEE Real Time on Embedded Technology and Applications Symposium, p.224-233, March 07-10, 2005 [doi>10.1109/RTAS.2005.40]
• 19: M. Angels Moncusí, Alex Arenas, and Jesus Labarta. Energy aware EDF scheduling in distributed hard real time systems. In Real-Time Systems Symposium, December 2003.
• 20: NAS Parallel Benchmark Suite v3.3.
• 21: PAPI: Performance application programming interface.
• 22: Barry Rountree , David K. Lowenthal , Shelby Funk , Vincent W. Freeh , Bronis R. de Supinski , Martin Schulz, Bounding energy consumption in large-scale MPI programs, Proceedings of the 2007 ACM/IEEE conference on Supercomputing, November 10-16, 2007, Reno, Nevada [doi>10.1145/1362622.1362688]
• 23: H. Saputra , M. Kandemir , N. Vijaykrishnan , M. J. Irwin , J. S. Hu , C-H. Hsu , U. Kremer, Energy-conscious compilation based on voltage scaling, Proceedings of the joint conference on Languages, compilers and tools for embedded systems: software and compilers for embedded systems, June 19-21, 2002, Berlin, Germany
• 24: Robert Springer , David K. Lowenthal , Barry Rountree , Vincent W. Freeh, Minimizing execution time in MPI programs on an energy-constrained, power-scalable cluster, Proceedings of the eleventh ACM SIGPLAN symposium on Principles and practice of parallel programming, March 29-31, 2006, New York, New York, USA [doi>10.1145/1122971.1123006]
• 25: Vishu Swaminathan , Krishnendu Chakrabarty, Investigating the effect of voltage-switching on low-energy task scheduling in hard real-time systems, Proceedings of the 2001 Asia and South Pacific Design Automation Conference, p.251, January 2001, Yokohama, Japan [doi>10.1145/370155.370337]
• 26: Vishnu Swaminathan and Krshnendu Chakrabarty. Real-time task scheduling for energy-aware embedded systems. In IEEE Real-Time Systems Symposium, November 2000.
• 27: OpenMPI Development Team. OpenMPI. http://www.open-mpi.org, 2006.
• 28: Ram Viswanath, Vijay Wakharkar, Abhay Watwe, and Vassou Lebonheur. Thermal performance challenges from silicon to systems. Intel Technology Journal, Q3 2000.
• 29: Yumin Zhang , Xiaobo Sharon Hu , Danny Z. Chen, Task scheduling and voltage selection for energy minimization, Proceedings of the 39th annual Design Automation Conference, June 10-14, 2002, New Orleans, Louisiana, USA [doi>10.1145/513918.513966]
• 30: Dakai Zhu , Rami Melhem , Bruce R. Childers, Scheduling with Dynamic Voltage/Speed Adjustment Using Slack Reclamation in Multiprocessor Real-Time Systems, IEEE Transactions on Parallel and Distributed Systems, v.14 n.7, p.686-700, July 2003 [doi>10.1109/TPDS.2003.1214320]

INDEX TERMS

Primary Classification:
  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.4 Processors
          Subjects: Run-time environments

General Terms:
Experimentation, Measurement, Performance

Keywords:
dvfs, dvs, energy, mpi, runtime