Scaling the bandwidth wall: challenges in and avenues for CMP scaling

ABSTRACT

As transistor density continues to grow at an exponential rate in accordance to Moore's law, the goal for many Chip Multi-Processor (CMP) systems is to scale the number of on-chip cores proportionally. Unfortunately, off-chip memory bandwidth capacity is projected to grow slowly compared to the desired growth in the number of cores. This creates a situation in which each core will have a decreasing amount of off-chip bandwidth that it can use to load its data from off-chip memory. The situation in which off-chip bandwidth is becoming a performance and throughput bottleneck is referred to as the bandwidth wall problem.

In this study, we seek to answer two questions: (1) to what extent does the bandwidth wall problem restrict future multicore scaling, and (2) to what extent are various bandwidth conservation techniques able to mitigate this problem. To address them, we develop a simple but powerful analytical model to predict the number of on-chip cores that a CMP can support given a limited growth in memory traffic capacity. We find that the bandwidth wall can severely limit core scaling. When starting with a balanced 8-core CMP, in four technology generations the number of cores can only scale to 24, as opposed to 128 cores under proportional scaling, without increasing the memory traffic requirement. We find that various individual bandwidth conservation techniques we evaluate have a wide ranging impact on core scaling, and when combined together, these techniques have the potential to enable super-proportional core scaling for up to 4 technology generations.

REFERENCES

• 1: Alaa R. Alameldeen , David A. Wood, Using compression to improve chip multiprocessor performance, University of Wisconsin at Madison, Madison, WI, 2006
• 2: Alaa R. Alameldeen , David A. Wood, Adaptive Cache Compression for High-Performance Processors, ACM SIGARCH Computer Architecture News, v.32 n.2, p.212, March 2004
• 3: A. R. Alameldeen and D. A. Wood. Frequent Pattern Compression: A Significance-Based Compression Scheme for L2 Caches. In Tech. Rep. 1500, Computer Sciences Department, University of Wisconsin-Madison, 2004.
• 4: Luiz André Barroso , Kourosh Gharachorloo , Edouard Bugnion, Memory system characterization of commercial workloads, Proceedings of the 25th annual international symposium on Computer architecture, p.3-14, June 27-July 02, 1998, Barcelona, Spain
• 5: C. Bienia, S. Kumar, J. P. Singh, and K. Li. The parsec suite: Characterization and architectural implications. Tech. Rep. TR-811-08, Princeton University, 2008.
• 6: Bryan Black , Murali Annavaram , Ned Brekelbaum , John DeVale , Lei Jiang , Gabriel H. Loh , Don McCaule , Pat Morrow , Donald W. Nelson , Daniel Pantuso , Paul Reed , Jeff Rupley , Sadasivan Shankar , John Shen , Clair Webb, Die Stacking (3D) Microarchitecture, Proceedings of the 39th Annual IEEE/ACM International Symposium on Microarchitecture, p.469-479, December 09-13, 2006 [doi>10.1109/MICRO.2006.18]
• 7: Doug Burger , James R. Goodman , Alain Kägi, Memory bandwidth limitations of future microprocessors, Proceedings of the 23rd annual international symposium on Computer architecture, p.78-89, May 22-24, 1996, Philadelphia, Pennsylvania, United States
• 8: Morris Marden, An Architectural Evaluation of Java TPC-W, Proceedings of the 7th International Symposium on High-Performance Computer Architecture, p.229, January 20-24, 2001
• 9: Chi F. Chen , Se-Hyun Yang , Babak Falsafi , Andreas Moshovos, Accurate and Complexity-Effective Spatial Pattern Prediction, Proceedings of the 10th International Symposium on High Performance Computer Architecture, p.276, February 14-18, 2004 [doi>10.1109/HPCA.2004.10010]
• 10: P. G. Emma , E. Kursun, Is 3D chip technology the next growth engine for performance improvement?, IBM Journal of Research and Development, v.52 n.6, p.541-552, November 2008
• 11: A. Hartstein, V. Srinivasan, T. Puzak, and P. Emma. On the Nature of Cache Miss Behavior: Is It p2? In The Journal of Instruction-Level Parallelism, volume 10, 2008.
• 12: Mark D. Hill , Michael R. Marty, Amdahl's Law in the Multicore Era, Computer, v.41 n.7, p.33-38, July 2008 [doi>10.1109/MC.2008.209]
• 13: Jaehyuk Huh , Doug Burger , Stephen W. Keckler, Exploring the Design Space of Future CMPs, Proceedings of the 2001 International Conference on Parallel Architectures and Compilation Techniques, p.199-210, September 08-12, 2001
• 14: ITRS. International Technology Roadmap for Semiconductors: 2005 Edition, Assembly and packaging. In http://www.itrs.net/Links/2005ITRS/AP2005.pdf, 2005.
• 15: Rakesh Kumar , Dean M. Tullsen , Norman P. Jouppi , Parthasarathy Ranganathan, Heterogeneous Chip Multiprocessors, Computer, v.38 n.11, p.32-38, November 2005 [doi>10.1109/MC.2005.379]
• 16: Rakesh Kumar , Dean M. Tullsen , Parthasarathy Ranganathan , Norman P. Jouppi , Keith I. Farkas, Single-ISA Heterogeneous Multi-Core Architectures for Multithreaded Workload Performance, Proceedings of the 31st annual international symposium on Computer architecture, p.64, June 19-23, 2004, München, Germany
• 17: Sanjeev Kumar , Christopher Wilkerson, Exploiting spatial locality in data caches using spatial footprints, Proceedings of the 25th annual international symposium on Computer architecture, p.357-368, June 27-July 02, 1998, Barcelona, Spain
• 18: H. Q. Le , W. J. Starke , J. S. Fields , F. P. O'Connell , D. Q. Nguyen , B. J. Ronchetti , W. M. Sauer , E. M. Schwarz , M. T. Vaden, IBM POWER6 microarchitecture, IBM Journal of Research and Development, v.51 n.6, p.639-662, November 2007
• 19: Y. Li, B. Lee, D. Brooks, Z. Hu, and K. Skadron. CMP design space exploration subject to physical constraints. In in 12th Intl. Symp. on High Performance Computer Architecture, 2006.
• 20: H. McGhan. Niagara 2 Opens the Floodgates. Microprocessor Report, 2006.
• 21: P. Pujara and A. Aggarwal. Increasing the cache efficiency by eliminating noise. High-Performance Computer Architecture, 2006. The Twelfth Intl. Symp. on, pages 145--154, 2006.
• 22: Kiran Puttaswamy , Gabriel H. Loh, Implementing Caches in a 3D Technology for High Performance Processors, Proceedings of the 2005 International Conference on Computer Design, p.525-532, October 02-05, 2005 [doi>10.1109/ICCD.2005.65]
• 23: Moinuddin K. Qureshi , M. Aater Suleman , Yale N. Patt, Line Distillation: Increasing Cache Capacity by Filtering Unused Words in Cache Lines, Proceedings of the 2007 IEEE 13th International Symposium on High Performance Computer Architecture, p.250-259, February 10-14, 2007 [doi>10.1109/HPCA.2007.346202]
• 24: Y. Solihin, F. Guo, T. R. Puzak, and P. G. Emma. Practical Cache Performance Modeling for Computer Architects. In Tutorial with HPCA--13, 2007.
• 25: Martin Thuresson , Lawrence Spracklen , Per Stenstrom, Memory-Link Compression Schemes: A Value Locality Perspective, IEEE Transactions on Computers, v.57 n.7, p.916-927, July 2008 [doi>10.1109/TC.2008.28]
• 26: Jessica H. Tseng , Hao Yu , Shailabh Nagar , Niteesh Dubey , Hubertus Franke , Pratap Pattnaik , Hiroshi Inoue , Toshio Nakatani, Performance Studies of Commercial Workloads on a Multi-core System, Proceedings of the 2007 IEEE 10th International Symposium on Workload Characterization, p.57-65, September 27-29, 2007 [doi>10.1109/IISWC.2007.4362181]
• 27: Tadaaki Yamauchi , Lance Hammond , and K Olukotun, A Single Chip Multiprocessor Integrated with High Density DRAM, Stanford University, Stanford, CA, 1997

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Modeling techniques

General Terms:
Measurement, Performance

Keywords:
analytical model, chip multi-processor, memory bandwidth