Hybrid cache architecture with disparate memory technologies

ABSTRACT

Caching techniques have been an efficient mechanism for mitigating the effects of the processor-memory speed gap. Traditional multi-level SRAM-based cache hierarchies, especially in the context of chip multiprocessors (CMPs), present many challenges in area requirements, core-to-cache balance, power consumption, and design complexity. New advancements in technology enable caches to be built from other technologies, such as Embedded DRAM (EDRAM), Magnetic RAM (MRAM), and Phase-change RAM (PRAM), in both 2D chips or 3D stacked chips. Caches fabricated in these technologies offer dramatically different power and performance characteristics when compared with SRAM-based caches, particularly in the areas of access latency, cell density, and overall power consumption. In this paper, we propose to take advantage of the best characteristics that each technology offers, through the use of Hybrid Cache Architecture (HCA) designs. We discuss and evaluate two types of hybrid cache architectures: inter cache Level HCA (LHCA), in which the levels in a cache hierarchy can be made of disparate memory technologies; and intra cache level or cache Region based HCA (RHCA), where a single level of cache can be partitioned into multiple regions, each of a different memory technology. We have studied a number of different HCA architectures and explored the potential of hardware support for intra-cache data movement and power consumption management within HCA caches. Utilizing a full-system simulator that has been validated against real hardware, we demonstrate that an LHCA design can provide a geometric mean 7% IPC improvement over a baseline 3-level SRAM cache design under the same area constraint across a collection of 25 workloads. A more aggressive RHCA-based design provides 12% IPC improvement over the baseline. Finally, a 2-layer 3D cache stack (3DHCA) of high density memory technology within the same chip footprint gives 18% IPC improvement over the baseline. Furthermore, up to 70% reduction in power consumption over a baseline SRAM-only design is achieved.

REFERENCES

• 1: D. A. Bader, Y. Li, T. Li, and V. Sachdeva. BioPerf: A Benchmark Suite to Evaluate High-performance Computer Architecture on Bioinformatics Applications. In Proceedings of the 2005 IEEE International Symposium on Workload Characterization, pages 163--173, 2005.
• 2: D. Bailey, J. Barton, T. Lasinski, and H. Simon. The NAS parallel benchmarks. In Technical report RNR-91-002 revision2, pages 453--464, 1991.
• 3: Bradford M. Beckmann , David A. Wood, Managing Wire Delay in Large Chip-Multiprocessor Caches, Proceedings of the 37th annual IEEE/ACM International Symposium on Microarchitecture, p.319-330, December 04-08, 2004, Portland, Oregon [doi>10.1109/MICRO.2004.21]
• 4: Christian Bienia , Sanjeev Kumar , Jaswinder Pal Singh , Kai Li, The PARSEC benchmark suite: characterization and architectural implications, Proceedings of the 17th international conference on Parallel architectures and compilation techniques, October 25-29, 2008, Toronto, Ontario, Canada [doi>10.1145/1454115.1454128]
• 5: Patrick Bohrer , James Peterson , Mootaz Elnozahy , Ram Rajamony , Ahmed Gheith , Ron Rockhold , Charles Lefurgy , Hazim Shafi , Tarun Nakra , Rick Simpson , Evan Speight , Kartik Sudeep , Eric Van Hensbergen , Lixin Zhang, Mambo: a full system simulator for the PowerPC architecture, ACM SIGMETRICS Performance Evaluation Review, v.31 n.4, p.8-12, March 2004 [doi>10.1145/1054907.1054910]
• 6: Bryan Black , Murali Annavaram , Ned Brekelbaum , John DeVale , Lei Jiang , Gabriel H. Loh , Don McCaule , Pat Morrow , Donald W. Nelson , Daniel Pantuso , Paul Reed , Jeff Rupley , Sadasivan Shankar , John Shen , Clair Webb, Die Stacking (3D) Microarchitecture, Proceedings of the 39th Annual IEEE/ACM International Symposium on Microarchitecture, p.469-479, December 09-13, 2006 [doi>10.1109/MICRO.2006.18]
• 7: Zeshan Chishti , Michael D. Powell , T. N. Vijaykumar, Optimizing Replication, Communication, and Capacity Allocation in CMPs, ACM SIGARCH Computer Architecture News, v.33 n.2, p.357-368, May 2005
• 8: L. Chung. Cell Design Considerations for Phase Change Memory as a Universal Memory. In International Symposium on VLSI Technology, Systems and Applications, pages 132--133, 2008.
• 9: W. Rhett Davis , John Wilson , Stephen Mick , Jian Xu , Hao Hua , Christopher Mineo , Ambarish M. Sule , Michael Steer , Paul D. Franzon, Demystifying 3D ICs: The Pros and Cons of Going Vertical, IEEE Design & Test, v.22 n.6, p.498-510, November 2005 [doi>10.1109/MDT.2005.136]
• 10: Xiangyu Dong , Xiaoxia Wu , Guangyu Sun , Yuan Xie , Helen Li , Yiran Chen, Circuit and microarchitecture evaluation of 3D stacking magnetic RAM (MRAM) as a universal memory replacement, Proceedings of the 45th annual Design Automation Conference, June 08-13, 2008, Anaheim, California [doi>10.1145/1391469.1391610]
• 11: Xiangyu Dong , Yuan Xie, System-level cost analysis and design exploration for three-dimensional integrated circuits (3D ICs), Proceedings of the 2009 Asia and South Pacific Design Automation Conference, January 19-22, 2009, Yokohama, Japan
• 12: Krisztián Flautner , Nam Sung Kim , Steve Martin , David Blaauw , Trevor Mudge, Drowsy caches: simple techniques for reducing leakage power, ACM SIGARCH Computer Architecture News, v.30 n.2, May 2002
• 13: S. Hanzawa, N. Kitai, K. Osada, A. Kotabe, Y. Matsui, N. Matsuzaki, N. Takaura, M. Moniwa, and T. Kawahara. A 512KB Embedded Phase Change Memory with 416kB/s Write Throughput at 100uA Cell Write Current. In IEEE International Solid-State Circuits Conference, pages 474--616, 2007.
• 14: M. Hosomi, H. Yamagishi, T. Yamamoto, and et al. A Novel Nonvolatile Memory with Spin Torque Transfer Magnetization Switching: Spin-RAM. In International Electron Devices Meeting, pages 459--462, 2005.
• 15: Jaehyuk Huh , Changkyu Kim , Hazim Shafi , Lixin Zhang , Doug Burger , Stephen W. Keckler, A NUCA substrate for flexible CMP cache sharing, Proceedings of the 19th annual international conference on Supercomputing, June 20-22, 2005, Cambridge, Massachusetts [doi>10.1145/1088149.1088154]
• 16: J. W. Joyner and J. D. Meindl. Opportunities for Reduced Power Dissipation Using Three-dimensional Integration. In Interconnect Technology Conference, pages 148--150, 2002.
• 17: Changkyu Kim , Doug Burger , Stephen W. Keckler, An adaptive, non-uniform cache structure for wire-delay dominated on-chip caches, Proceedings of the 10th international conference on Architectural support for programming languages and operating systems, October 05-09, 2002, San Jose, California
• 18: Feihui Li , Chrysostomos Nicopoulos , Thomas Richardson , Yuan Xie , Vijaykrishnan Narayanan , Mahmut Kandemir, Design and Management of 3D Chip Multiprocessors Using Network-in-Memory, Proceedings of the 33rd annual international symposium on Computer Architecture, p.130-141, June 17-21, 2006
• 19: Christianto C. Liu , Ilya Ganusov , Martin Burtscher , Sandip Tiwari, Bridging the Processor-Memory Performance Gapwith 3D IC Technology, IEEE Design & Test, v.22 n.6, p.556-564, November 2005 [doi>10.1109/MDT.2005.134]
• 20: Gabriel H. Loh, 3D-Stacked Memory Architectures for Multi-core Processors, Proceedings of the 35th International Symposium on Computer Architecture, p.453-464, June 21-25, 2008
• 21: N. Madan, L. Zhao, N. Muralimanohar, A. Udipi, R. Balasubramonian, R. Iyer, S. Makineni, and D. Newell. Optimizing communication and capacity in a 3D stacked reconfigurable cache hierarchy. In High Performance Computer Architecture, pages 262--274, Feb. 2009.
• 22: R. Morin, A. Kumar, and E. Ilyina. A multi-level comparative performance characterization of specjbb2005 versus specjbb2000. In Proceedings of the IEEE International Workload Characterization, pages 67--75, Oct. 2005.
• 23: Naveen Muralimanohar , Rajeev Balasubramonian , Norm Jouppi, Optimizing NUCA Organizations and Wiring Alternatives for Large Caches with CACTI 6.0, Proceedings of the 40th Annual IEEE/ACM International Symposium on Microarchitecture, p.3-14, December 01-05, 2007 [doi>10.1109/MICRO.2007.30]
• 24: F. Pellizzer, A. Pirovano, F. Ottogalli, M. Magistretti, M. Scaravaggi, P. Zuliani, M. Tosi, A. Benvenuti, P. Besana, S. Cadeo, T. Marangon, R. Morandi, R. Piva, A. Spandre, R. Zonca, A. Modelli, E. Varesi, T. Lowrey, A. Lacaita, G. Casagrande, P. Cappelletti, and R. Bez. Novel utrench Phase-change Memory Cell for Embedded and Stand-alone Non-volatile Memory Applications. In Symposium on VLSI Technology, pages 18--19, 2004.
• 25: B. Sinharoy , R. N. Kalla , J. M. Tendler , R. J. Eickemeyer , J. B. Joyner, POWER5 System microarchitecture, IBM Journal of Research and Development, v.49 n.4/5, p.505-521, July 2005
• 26: SPEC. Standard Performance Evaluation Corporation. http://www.spec.org/cpu2006/. 2006.
• 27: G. Sun, X. Dong, Y. Xie, J. Li, and Y. Chen. A novel architecture of the 3D stacked MRAM L2 cache for CMPs. In High Performance Computer Architecture, pages 239--249, Feb. 2009.
• 28: X. Wu, J. Li, L. Zhang, E. Speight, and Y. Xie. Power and Performance of Read-Write Aware Hybrid Caches with Non-volatile Memories. In Design, Automation and Test in Europe, 2009.
• 29: Yuan Xie , Gabriel H. Loh , Bryan Black , Kerry Bernstein, Design space exploration for 3D architectures, ACM Journal on Emerging Technologies in Computing Systems (JETC), v.2 n.2, p.65-103, April 2006 [doi>10.1145/1148015.1148016]
• 30: W. Zhao, E. Belhaire, Q. Mistral, C. Chappert, V. Javerliac, B. Dieny, and E. Nicolle. Macro-model of Spin-Transfer Torque based Magnetic Tunnel Junction device for hybrid Magnetic--CMOS design. In IEEE International Behavioral Modeling and Simulation Workshop, pages 40--43, 2006.

INDEX TERMS

Primary Classification:
  B. Hardware
  B.3 MEMORY STRUCTURES
      B.3.2 Design Styles
          Subjects: Cache memories

General Terms:
Design, Experimentation, Performance

Keywords:
hybrid cache architecture, three-dimensional ic