The case for simple, visible cache coherency

ABSTRACT

The shared memory research community has proposed many complex communication protocols that aim to eliminate specific performance bottlenecks, while still providing an easy-to-use communication interface. Although tailored protocols can eliminate some bottlenecks that arise in real applications, removing the cause of the bottleneck through software optimizations and bug fixes is cheaper to implement, faster to fix (once found), and requires no additional support by the hardware beyond a simple shared memory interface. In fact, in our experience, the choice of coherence protocol is much less important than providing an efficient hardware feedback that indentifies the source of the problem. Future cache-coherence research should focus efforts on illuminating memory system behavior, providing smarter tools to identify bottlenecks, and helping to eliminate them through software optimizations.

REFERENCES

• 1: Vishal Aslot , Max J. Domeika , Rudolf Eigenmann , Greg Gaertner , Wesley B. Jones , Bodo Parady, SPEComp: A New Benchmark Suite for Measuring Parallel Computer Performance, Proceedings of the International Workshop on OpenMP Applications and Tools: OpenMP Shared Memory Parallel Programming, p.1-10, July 30-31, 2001
• 2: L. M. Censier , P. Feautrier, A New Solution to Coherence Problems in Multicache Systems, IEEE Transactions on Computers, v.27 n.12, p.1112-1118, December 1978 [doi>10.1109/TC.1978.1675013]
• 3: David Chaiken , John Kubiatowicz , Anant Agarwal, LimitLESS directories: A scalable cache coherence scheme, Proceedings of the fourth international conference on Architectural support for programming languages and operating systems, p.224-234, April 08-11, 1991, Santa Clara, California, United States
• 4: Babak Falsafi , David A. Wood, Reactive NUMA: a design for unifying S-COMA and CC-NUMA, Proceedings of the 24th annual international symposium on Computer architecture, p.229-240, June 01-04, 1997, Denver, Colorado, United States
• 5: Jeffrey Steven Gibson , John Hennessy, Memory profiling on shared-memory multiprocessors, Stanford University, Stanford, CA, 2002
• 6: Jeff Gibson , Robert Kunz , David Ofelt , Mark Horowitz , John Hennessy , Mark Heinrich, FLASH vs. (Simulated) FLASH: closing the simulation loop, Proceedings of the ninth international conference on Architectural support for programming languages and operating systems, p.49-58, November 2000, Cambridge, Massachusetts, United States
• 7: Erik Hagersten , Michael Koster, WildFire: A Scalable Path for SMPs, Proceedings of the 5th International Symposium on High Performance Computer Architecture, p.172, January 09-12, 1999
• 8: David V. James , Anthony T. Laundrie , Stein Gjessing , Gurindar Sohi, Scalable coherent interface, Computer, v.23 n.6, p.74-77, June 1990 [doi>10.1109/2.55503]
• 9: R. Kalla, B. Sinharoy, and J. Tendler. Simultaneous Multithreading Implementations in POWER5---IBM's Next Generation POWER Microprocessor. In Hot Chips 15, August 2003.
• 10: S. Kapil. Gemini: A Power-efficient Chip Multi-Threaded (CMT) UltraSPARC Processor. In Hot Chips 15, August 2003.
• 11: P. Kongetiraer. A 32-way Multithreaded SPARC processor. In Hot Chips 16, August 2004.
• 12: R. Kunz. Performance Bottlenecks on Large-Scale Shared-Memory Multiprocessors. PhD thesis, Stanford University, 2005.
• 13: J. Kuskin , D. Ofelt , M. Heinrich , J. Heinlein , R. Simoni , K. Gharachorloo , J. Chapin , D. Nakahira , J. Baxter , M. Horowitz , A. Gupta , M. Rosenblum , J. Hennessy, The Stanford FLASH multiprocessor, Proceedings of the 21st annual international symposium on Computer architecture, p.302-313, April 18-21, 1994, Chicago, Illinois, United States
• 14: An-Chow Lai , Babak Falsafi, Memory sharing predictor: the key to a speculative coherent DSM, Proceedings of the 26th annual international symposium on Computer architecture, p.172-183, May 01-04, 1999, Atlanta, Georgia, United States
• 15: James Laudon , Daniel Lenoski, The SGI Origin: a ccNUMA highly scalable server, Proceedings of the 24th annual international symposium on Computer architecture, p.241-251, June 01-04, 1997, Denver, Colorado, United States
• 16: Daniel Lenoski , James Laudon , Truman Joe , David Nakahira , Luis Stevens , Anoop Gupta , John Hennessy, The DASH prototype: implementation and performance, Proceedings of the 19th annual international symposium on Computer architecture, p.92-103, May 19-21, 1992, Queensland, Australia
• 17: Njuguna Njoroge , Jared Casper , Sewook Wee , Yuriy Teslyar , Daxia Ge , Christos Kozyrakis , Kunle Olukotun, ATLAS: a chip-multiprocessor with transactional memory support, Proceedings of the conference on Design, automation and test in Europe, April 16-20, 2007, Nice, France
• 18: A. Nowatzyk et al. S3.mp: A Multiprocessor in a Matchbox. In Proceedings of PASA, June 1993.
• 19: S. K. Reinhardt , J. R. Larus , D. A. Wood, Tempest and typhoon: user-level shared memory, Proceedings of the 21st annual international symposium on Computer architecture, p.325-336, April 18-21, 1994, Chicago, Illinois, United States
• 20: A. Saulsbury , T. Wilkinson , J. Carter , A. Landin, An argument for simple COMA, Proceedings of the 1st IEEE Symposium on High-Performance Computer Architecture, p.276, January 22-25, 1995
• 21: Daniel J. Scales , Kourosh Gharachorloo , Chandramohan A. Thekkath, Shasta: a low overhead, software-only approach for supporting fine-grain shared memory, Proceedings of the seventh international conference on Architectural support for programming languages and operating systems, p.174-185, October 01-04, 1996, Cambridge, Massachusetts, United States
• 22: Daniel J. Sorin , Vijay S. Pai , Sarita V. Adve , Mary K. Vernon , David A. Wood, Analytic evaluation of shared-memory systems with ILP processors, Proceedings of the 25th annual international symposium on Computer architecture, p.380-391, June 27-July 02, 1998, Barcelona, Spain
• 23: Vijayaraghavan Soundararajan , Mark Heinrich , Ben Verghese , Kourosh Gharachorloo , Anoop Gupta , John Hennessy, Flexible use of memory for replication/migration in cache-coherent DSM multiprocessors, Proceedings of the 25th annual international symposium on Computer architecture, p.342-355, June 27-July 02, 1998, Barcelona, Spain

INDEX TERMS

Primary Classification:
  B. Hardware
  B.8 Performance and Reliability
      B.8.2 Performance Analysis and Design Aids

Additional Classification:
  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.3 Concurrent Programming

General Terms:
Measurement, Performance

Keywords:
FLASH, SpecOMP2001, cc-NUMA, coherence protocol, performance bottlenecks, shared-memory multiprocessor, software optimization