JouleSort: a balanced energy-efficiency benchmark

ABSTRACT

The energy efficiency of computer systems is an important concern in a variety of contexts. In data centers, reducing energy use improves operating cost, scalability, reliability, and other factors. For mobile devices, energy consumption directly affects functionality and usability. We propose and motivate JouleSort, an external sort benchmark, for evaluating the energy efficiency of a wide range of computer systems from clusters to handhelds. We list the criteria, challenges, and pitfalls from our experience in creating a fair energy-efficiency benchmark. Using a commercial sort, we demonstrate a JouleSort system that is over 3.5x as energy-efficient as last year's estimated winner. This system is quite different from those currently used in data centers. It consists of a commodity mobile CPU and 13 laptop drives connected by server-style I/O interfaces.

REFERENCES

• 1: Anonymous et al. A measure of transaction processing performance. In Datamation, pages 112--118, Apr. 1985.
• 2: Luiz André Barroso, The price of performance, Queue, v.3 n.7, September 2005 [doi>10.1145/1095408.1095420]
• 3: P. Bose. Keynote address: Power-efficient microarchitectural choices at the early definition stage. In PACS, 2003.
• 4: D. Brooks and M. Martonosi. Dynamic thermal management for high-performance microprocessors. In HPCA, 2001.
• 5: C. Calwell and T. Reeder. Power supplies: A hidden opportunity for energy savings. Online, May 2002. http://www.ecosconsulting.com/resourcespublications.html#PowerSupply.
• 6: J. Chase, D. Anderson, et al. Managing energy and server resources in hosting centers. In SOSP, 2001.
• 7: J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. In OSDI, Dec. 2004.
• 8: F. Douglis, P. Krishnan, and B. Marsh. Thwarting the power-hungry disk. In USENIX, 1994.
• 9: Embedded Microprocessor Benchmark Consortium (EEMBC). EnergyBench benchmark software. Online.http://www.eembc.org/benchmark/power\_sl.asp.
• 10: X. Fan, C. Ellis, and A. Lebeck. Memory controller policies for DRAM power management. In Low-Power Systems and Design (ISLPED), 2001.
• 11: Wes Felter , Karthick Rajamani , Tom Keller , Cosmin Rusu, A performance-conserving approach for reducing peak power consumption in server systems, Proceedings of the 19th annual international conference on Supercomputing, June 20-22, 2005, Cambridge, Massachusetts [doi>10.1145/1088149.1088188]
• 12: J. Flinn and M. Satyanarayanan. Energy-aware adaptation for mobile applications. In SOSP, 1999.
• 13: R. Gonzalez and M. Horowitz. Energy dissipation in general purpose microprocessors. IEEE Journal of Solid-State Circuits, 31(9):1277--1284, 1996.
• 14: K. Govil, E. Chen, and H. Wasserman. Comparing algorithms for dynamic speed-setting of a low-power CPU. In MobiCom, 1995.
• 15: N. K. Govindaraju, J. Gray, R. Kumar, and D. Manocha. GPUTeraSort: High performance graphics coprocessor sorting for large database management. In SIGMOD, June 2006.
• 16: J. Gray. Sort benchmark home page, Oct. 2006. http://research.microsoft.com/barc/SortBenchmark.
• 17: J. Gray, J. Coates, and C. Nyberg. Performance/Price Sort and PennySort. Technical Report MS-TR-98-45, Microsoft, Aug. 1998.
• 18: J. Gray and B. Fitzgerald. Flash disk opportunity for server-applications. Online, 2007. http://www.microsoft.com/gray.
• 19: HP enterprise configurator power calculators, Oct. 2006. http://h30099.www3.hp.com/configurator/powercalcs.asp.
• 20: Christine E. Jones , Krishna M. Sivalingam , Prathima Agrawal , Jyh Cheng Chen, A Survey of Energy Efficient Network Protocols for Wireless Networks, Wireless Networks, v.7 n.4, p.343-358, 08/01/2001 [doi>10.1023/A:1016627727877]
• 21: James Laudon, Performance/Watt: the new server focus, ACM SIGARCH Computer Architecture News, v.33 n.4, November 2005 [doi>10.1145/1105734.1105737]
• 22: M. R. Lindeburg. Mechanical Engineering Reference Manual. Professional Publications, tenth edition, 1997.
• 23: P. Lyman and H. R. Varian. How much information? Online, 2003. http://www2.sims.berkeley.edu/research/projects/how-much-info-2003/.
• 24: J. Markoff and S. Hansell. Hiding in plain sight, Google seeks an expansion of power. New York Times. June 14, 2006.
• 25: Chris Nyberg , Tom Barclay , Zarka Cvetanovic , Jim Gray , Dave Lomet, AlphaSort: a cache-sensitive parallel external sort, The VLDB Journal — The International Journal on Very Large Data Bases, v.4 n.4, October 1995
• 26: C. Nyberg and C. Koester. Ordinal Technology-NSort Home Page. Online, 2007. http://www.ordinal.com/.
• 27: C. D. Patel. A vision of energy aware computing from chips to data centers. In Micro-Mechanical Engineering (ISMME), Dec. 2003.
• 28: C. D. Patel and P. Ranganathan. Enterprise power and cooling. ASPLOS Tutorial, Oct. 2006.
• 29: P. Pillai and K. G. Shin. Real-time dynamic voltage scaling for low-power embedded operating systems. In SOSP, pages 89--102, 2001.
• 30: E. Pinheiro, R. Bianchini, et al. Load balancing and unbalancing for power and performance in cluster-based systems. In Workshop on Compilers and Operating Systems for Low Power (COLP), 2001.
• 31: Parthasarathy Ranganathan , Erik Geelhoed , Meera Manahan , Ken Nicholas, Energy-Aware User Interfaces and Energy-Adaptive Displays, Computer, v.39 n.3, p.31-38, March 2006 [doi>10.1109/MC.2006.89]
• 32: P. Ranganathan, P. Leech, et al. Ensemble-level power management for dense blade servers. In ISCA, 2006.
• 33: Standard Performance Evaluation Corporation (SPEC). SPEC power and performance committee. Online.http://www.spec.org/specpower/.
• 34: Sun Microsystems. SWaP (space, watts and performance) metric. Online. http://www.sun.com/servers/coolthreads/swap/.
• 35: United States Environmental Protection Agency (EPA). Enterprise server and data center efficiency initiatives. Online. http://www.energystar.gov/index.cfm? c=products.pr\_servers\_datacenters.
• 36: A. Weissel and F. Bellosa. Process cruise control: event-driven clock scaling for dynamic power management. In Compilers, Architecture, and Synthesis for Embedded Systems (CASES), 2002.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.4 Systems

General Terms:
Design, Experimentation, Measurement, Performance

Keywords:
benchmark, energy-efficiency, power, servers, sort