Compiler managed micro-cache bypassing for high performance EPIC processors

ABSTRACT

Advanced microprocessors have been increasing clock rates, well beyond the Gigahertz boundary. For such high performance microprocessors, a small and fast data micro cache (ucache) is important to overall performance, and proper management of it via load bypassing has a significant performance impact. In this paper, we propose and evaluate a hardware-software collaborative technique to manage ucache bypassing for EPIC processors. The hardware supports the ucache bypassing with a flag in the load instruction format, and the compiler employs static analysis and profiling to identify loads that should bypass the ucache. The collaborative method achieves a significant improvement in performance for the SpecInt2000 benchmarks. On average, about 40%, 30%, 24%, and 22% of load references are identified to bypass 256B, 1K, 4K, and 8K sized ucaches, respectively. This reduces the ucache miss rates by 39%, 32%, 28%, and 26%. The number of pipeline stalls from loads to their uses is reduced by 13%, 9%, 6%, and 5%. Meanwhile, the L1 and L2 cache misses remain largely unchanged. For the 256B ucache, bypassing improves overall performance on average by 5%.

REFERENCES

• 1: Jay Bharadwaj , Kishore Menezes , Chris McKinsey, Wavefront scheduling: path based data representation and scheduling of subgraphs, Proceedings of the 32nd annual ACM/IEEE international symposium on Microarchitecture, p.262-271, November 16-18, 1999, Haifa, Israel
• 2: Chi, C. H.; Dietz, H., "Improving cache performance by selective cache bypass," Proceedings of the Twenty-Second Annual Hawaii International Conference on System Sciences, 1989. Vol. I: Architecture Track, 1989, pp 277--285 vol. 1
• 3: John Crawford, Guest Editor's Introduction: Introducing the Itanium Processors, IEEE Micro, v.20 n.5, p.9-11, September 2000 [doi>10.1109/MM.2000.877946]
• 4: Brian Fields , Shai Rubin , Rastislav Bodík, Focusing processor policies via critical-path prediction, Proceedings of the 28th annual international symposium on Computer architecture, p.74-85, June 30-July 04, 2001, Göteborg, Sweden
• 5: Brian Fields , Rastislav Bodík , Mark D. Hill, Slack: maximizing performance under technological constraints, ACM SIGARCH Computer Architecture News, v.30 n.2, May 2002
• 6: Brian R. Fisk, The Non-Critical Buffer: Using Load Latency Tolerance to Improve Data Cache Efficiency, Proceedings of the 1999 IEEE International Conference on Computer Design, p.538, October 10-13, 1999
• 7: David M. Gallagher , William Y. Chen , Scott A. Mahlke , John C. Gyllenhaal , Wen-mei W. Hwu, Dynamic memory disambiguation using the memory conflict buffer, ACM SIGPLAN Notices, v.29 n.11, p.183-193, Nov. 1994
• 8: Richard Johnson , Keshav Pingali, Dependence-based program analysis, Proceedings of the ACM SIGPLAN 1993 conference on Programming language design and implementation, p.78-89, June 21-25, 1993, Albuquerque, New Mexico, United States
• 9: Teresa L. Johnson , Daniel A. Connors , Matthew C. Merten , Wen-mei W. Hwu, Run-Time Cache Bypassing, IEEE Transactions on Computers, v.48 n.12, p.1338-1354, December 1999 [doi>10.1109/12.817393]
• 10: Kurd, N.A.; Barkarullah, J.S.; Dizon, R.O.; Fletcher, T.D.; Madland, P.D. "A multigigahertz clocking scheme for the Pentium(R) 4 microprocessor," IEEE Journal of Solid-State Circuits, Volume: 36 Issue: 11, Nov. 2001 pp 1647--1653
• 11: Alvin R. Lebeck , David A. Wood, Cache Profiling and the SPEC Benchmarks: A Case Study, Computer, v.27 n.10, p.15-26, October 1994 [doi>10.1109/2.318580]
• 12: Hsien-Hsin S. Lee , Gary S. Tyson, Region-based caching: an energy-delay efficient memory architecture for embedded processors, Proceedings of the 2000 international conference on Compilers, architecture, and synthesis for embedded systems, p.120-127, November 17-19, 2000, San Jose, California, United States [doi>10.1145/354880.354898]
• 13: Livadas, P.E.; Roy, P.K., "Program dependence analysis," Proceedings of Conference on Software Maintenance, 1992, pp 356--365
• 14: Scott A. Mahlke , William Y. Chen , Roger A. Bringmann , Richard E. Hank , Wen-Mei W. Hwu , B. Ramakrishna Rau , Michael S. Schlansker, Sentinel scheduling: a model for compiler-controlled speculative execution, ACM Transactions on Computer Systems (TOCS), v.11 n.4, p.376-408, Nov. 1993 [doi>10.1145/161541.159765]
• 15: Pyreddy R. and G. Tyson, "Exploiting Load Latency Tolerance for Relaxing Cache Design Constraints," Workshop on Complexity-Effective Design (WCED), Anchorage, Alaska, USA, May 25--26, 2002
• 16: Rakvic, Ryan; Black, Bryan; Limaye, Deepak; and Shen, John. "Non-vital Loads," HPCA 8, Feb 2--6, 2002.
• 17: Michael S. Schlansker , B. Ramakrishna Rau, EPIC: Explicitly Parallel Instruction Computing, Computer, v.33 n.2, p.37-45, February 2000 [doi>10.1109/2.820037]
• 18: Harsh Sharangpani , Ken Arora, Itanium Processor Microarchitecture, IEEE Micro, v.20 n.5, p.24-43, September 2000 [doi>10.1109/40.877948]
• 19: Srikanth T. Srinivasan , Roy Dz-ching Ju , Alvin R. Lebeck , Chris Wilkerson, Locality vs. criticality, Proceedings of the 28th annual international symposium on Computer architecture, p.132-143, June 30-July 04, 2001, Göteborg, Sweden
• 20: Artour Stoutchinin , José N. Amaral , Guang R. Gao , James C. Dehnert , Suneel Jain , Alban Douillet, Speculative Prefetching of Induction Pointers, Proceedings of the 10th International Conference on Compiler Construction, p.289-303, April 02-06, 2001
• 21: Edward S. Tam , Jude A. Rivers , Vijayalakshmi Srinivasan , Gary S. Tyson , Edward S. Davidson, Active Management of Data Caches by Exploiting Reuse Information, IEEE Transactions on Computers, v.48 n.11, p.1244-1259, November 1999 [doi>10.1109/12.811113]
• 22: Dongning Liang, Dynamic Prediction of Critical Path Instructions, Proceedings of the 7th International Symposium on High-Performance Computer Architecture, p.185, January 20-24, 2001
• 23: Gary Tyson , Matthew Farrens , John Matthews , Andrew R. Pleszkun, A modified approach to data cache management, Proceedings of the 28th annual international symposium on Microarchitecture, p.93-103, November 29-December 01, 1995, Ann Arbor, Michigan, United States
• 24: Uhlig, R.; R. Fishtein; O. Gershon; I. Hirsh; H. Wang, "SoftSDV: A Pre-silicon Software Development Environment for the IA-64 Architecture", Intel Technology Journal Q4 1999, http://www.intel.com/technology/itj/q41999/articles/art_2.htm
• 25: Youfeng Wu, Efficient discovery of regular stride patterns in irregular programs and its use in compiler prefetching, Proceedings of the ACM SIGPLAN 2002 Conference on Programming language design and implementation, June 17-19, 2002, Berlin, Germany

INDEX TERMS

Primary Classification:
  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.4 Processors
          Subjects: Compilers

Additional Classification:
  B. Hardware
  B.3 MEMORY STRUCTURES
      B.3.2 Design Styles
          Subjects: Cache memories
  B.7 INTEGRATED CIRCUITS
      B.7.1 Types and Design Styles
          Subjects: Microprocessors and microcomputers

  C. Computer Systems Organization
  C.5 COMPUTER SYSTEM IMPLEMENTATION
      C.5.3 Microcomputers
          Subjects: Microprocessors

General Terms:
Performance