The implications of working set analysis on supercomputing memory hierarchy design

ABSTRACT

Supercomputer architects strive to maximize the performance of scientific applications. Unfortunately, the large, unwieldy nature of most scientific applications has lead to the creation of artificial benchmarks, such as SPEC-FP, for architecture research. Given the impact that these benchmarks have on architecture research, this paper seeks an understanding of how they relate to real-world applications within the Department of Energy. Since the memory system has been found to be a particularly key issue for many applications, the focus of the paper is on the relationship between how the SPEC-FP benchmarks and DOE applications use the memory system. The results indicate that while the SPEC-FP suite is a well balanced suite, supercomputing applications typically demand more from the memory system and must perform more "other work" (in the form of integer computations) along with the floating point operations. The SPEC-FP suite generally demonstrates slightly more temporal locality leading to somewhat lower bandwidth demands. The most striking result is the cumulative difference between the benchmarks and the applications in terms of the requirements to sustain the floating-point operation rate: the DOE applications require significantly more data from main memory (not cache) per FLOP and dramatically more integer instructions per FLOP.

REFERENCES

• 1: ASC Purple benchmark codes, July 2004. http://www.llnl.gov/asci/purple/benchmarks/limited/code_list.html.
• 2: SPEC website, July 2004. http://www.spec.org.
• 3: Apple Architecture Performance Groups. Computer Hardware Understanding Development Tools 2.0 Reference Guide for MacOS X. Apple Computer Inc, July 2002.
• 4: Vishal Aslot , Rudolf Eigenmann, Performance characteristics of the SPEC OMP2001 benchmarks, ACM SIGARCH Computer Architecture News, v.29 n.5, December 2001 [doi>10.1145/563647.563655]
• 5: Luiz André Barroso , Kourosh Gharachorloo , Edouard Bugnion, Memory system characterization of commercial workloads, Proceedings of the 25th annual international symposium on Computer architecture, p.3-14, June 27-July 02, 1998, Barcelona, Spain
• 6: IEEE Micro staff, The Use and Abuse of SPEC: An ISCA Panel, IEEE Micro, v.23 n.4, p.73-77, July 2003 [doi>10.1109/MM.2003.1225977]
• 7: Z. Cvetanovic , D. Bhandarkar, Characterization of alpha AXP performance using TP and SPEC workloads, Proceedings of the 21ST annual international symposium on Computer architecture, p.60-70, April 18-21, 1994, Chicago, Illinois, United States
• 8: Peter J. Denning, The working set model for program behavior, Proceedings of the first ACM symposium on Operating System Principles, p.15.1-15.12, January 1967 [doi>10.1145/800001.811670]
• 9: Domenico Ferrari, A generative model of working set dynamics, Proceedings of the 1981 ACM SIGMETRICS conference on Measurement and modeling of computer systems, p.52-57, September 14-16, 1981, Las Vegas, Nevada, United States
• 10: Jeffrey D. Gee , Mark D Hill , Alan J Smith, Cache Performance of the SPEC Benchmark Suite, University of California at Berkeley, Berkeley, CA, 1991
• 11: Somnath Ghosh , Margaret Martonosi , Sharad Malik, Cache miss equations: an analytical representation of cache misses, Proceedings of the 11th international conference on Supercomputing, p.317-324, July 07-11, 1997, Vienna, Austria [doi>10.1145/263580.263657]
• 12: Swathi Tanjore Gurumani , Aleksandar Milenkovic, Execution characteristics of SPEC CPU2000 benchmarks: Intel C++ vs. Microsoft VC++, Proceedings of the 42nd annual Southeast regional conference, April 02-03, 2004, Huntsville, Alabama [doi>10.1145/986537.986599]
• 13: Magnus Karlsson , Per Stenström, An analytical model of the working-set sizes in decision-support systems, Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.275-285, June 18-21, 2000, Santa Clara, California, United States
• 14: Kimberly Keeton , David A. Patterson , Yong Qiang He , Roger C. Raphael , Walter E. Baker, Performance characterization of a Quad Pentium Pro SMP using OLTP workloads, Proceedings of the 25th annual international symposium on Computer architecture, p.15-26, June 27-July 02, 1998, Barcelona, Spain
• 15: Dennis C. Lee , Patrick J. Crowley , Jean-Loup Baer , Thomas E. Anderson , Brian N. Bershad, Execution characteristics of desktop applications on Windows NT, Proceedings of the 25th annual international symposium on Computer architecture, p.27-38, June 27-July 02, 1998, Barcelona, Spain
• 16: Jack L. Lo , Luiz André Barroso , Susan J. Eggers , Kourosh Gharachorloo , Henry M. Levy , Sujay S. Parekh, An analysis of database workload performance on simultaneous multithreaded processors, Proceedings of the 25th annual international symposium on Computer architecture, p.39-50, June 27-July 02, 1998, Barcelona, Spain
• 17: Ann Marie Grizzaffi Maynard , Colette M. Donnelly , Bret R. Olszewski, Contrasting characteristics and cache performance of technical and multi-user commercial workloads, Proceedings of the sixth international conference on Architectural support for programming languages and operating systems, p.145-156, October 05-07, 1994, San Jose, California, United States
• 18: McCalpin, John D. Stream: Sustainable memory bandwidth in high performance computers, 1997.
• 19: Sally A. McKee, Reflections on the memory wall, Proceedings of the 1st conference on Computing frontiers, April 14-16, 2004, Ischia, Italy [doi>10.1145/977091.977115]
• 20: SPEC Open Systems Steering Committee. Spec cpu 2000 run and reporting rules (revised). March 15, 2001.
• 21: Steve Plimpton, Fast parallel algorithms for short-range molecular dynamics, Journal of Computational Physics, v.117 n.1, p.1-19, March 1, 1995 [doi>10.1006/jcph.1995.1039]
• 22: Steven J. Plimpton. Lammps web page, July 2004. http://www.cs.sandia.gov/ sjplimp/lammps.html.
• 23: Steven J. Plimpton, R. Pollock, and M, Stevens. Particle-mesh ewald and rRESPA for parallel molecular dynamics. In Proceedings of the Eighth SIAM Conference on Parallel Processing for Scientific Computing, Minneapolis, MN, March 1997.
• 24: Arun Rodrigues , Richard Murphy , Peter Kogge , Keith Underwood, Characterizing a new class of threads in scientific applications for high end supercomputers, Proceedings of the 18th annual international conference on Supercomputing, June 26-July 01, 2004, Malo, France [doi>10.1145/1006209.1006234]
• 25: Juan Rodriguez-Rosell, Empirical working set behavior, Communications of the ACM, v.16 n.9, p.556-560, Sept. 1973 [doi>10.1145/362342.362356]
• 26: Edward Rothberg , Jaswinder Pal Singh , Anoop Gupta, Working sets, cache sizes, and node granularity issues for large-scale multiprocessors, Proceedings of the 20th annual international symposium on Computer architecture, p.14-26, May 16-19, 1993, San Diego, California, United States
• 27: Timothy Sherwood , Erez Perelman , Greg Hamerly , Brad Calder, Automatically characterizing large scale program behavior, Proceedings of the 10th international conference on Architectural support for programming languages and operating systems, October 05-09, 2002, San Jose, California
• 28: D. R. Slutz , I. L. Traiger, A note on the calculation of average working set size, Communications of the ACM, v.17 n.10, p.563-565, Oct. 1974 [doi>10.1145/355620.361167]
• 29: Deepu Talla , Lizy Kurian John , Doug Burger, Bottlenecks in Multimedia Processing with SIMD Style Extensions and Architectural Enhancements, IEEE Transactions on Computers, v.52 n.8, p.1015-1031, August 2003 [doi>10.1109/TC.2003.1223637]
• 30: Steven Cameron Woo , Moriyoshi Ohara , Evan Torrie , Jaswinder Pal Singh , Anoop Gupta, The SPLASH-2 programs: characterization and methodological considerations, Proceedings of the 22nd annual international symposium on Computer architecture, p.24-36, June 22-24, 1995, S. Margherita Ligure, Italy

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS

Keywords:
performance modeling, supercomputing, working set