Leveraging Optical Technology in Future Bus-based Chip Multiprocessors

ABSTRACT

Although silicon optical technology is still in its formative stages, and the more near-term application is chip-to-chip communication, rapid advances have been made in the development of on-chip optical interconnects. In this paper, we investigate the integration of CMOS-compatible optical technology to on-chip cache-coherent buses in future CMPs. While not exhaustive, our investigation yields a hierarchical opto-electrical system that exploits the advantages of optical technology while abiding by projected limitations. Our evaluation shows that, for the applications considered, compared to an aggressive all-electrical bus of similar power and area, significant performance improvements can be achieved using an opto-electrical bus. This performance improvement is largely dependent on the application's bandwidth demand and on the number of implemented wavelengths per optical waveguide. We also present a number of critical areas for future work that we discover in the course of our research.

REFERENCES

• 1: [1] V. R. Almeida, C. A. Barrios, R. R. Panepucci, M. Lipson, M.A. Foster, D. G. Ouzounov, and A. L. Gaeta. All-optical switching on a silicon chip. Optics Letters, 29(24):2867, December 2004.
• 2: [2] H. B. Bakoglu. Circuits, Interconnections, and Packaging for VLSI. Addison-Wesley, Menlo Park, CA, 1990.
• 3: [3] K. Banerjee and A. Mehrotra. Power dissipation issues in interconnect performance optimization for sub-180nm designs. In Symposium on VLSI Circuits Digest of Technical Papers, pages 12-15, Honolulu, June 2002.
• 4: [4] C. A. Barrios, V. R. de Almeida, and M. Lipson. Low-power-consumption short-length and high-modulation-depth silicon electrooptic modulator. Journal of Lightwave Technology, 21(4):1089-1098, April 2003.
• 5: Luis André Barroso , Michel Dubois, The performance of cache-coherent ring-based multiprocessors, Proceedings of the 20th annual international symposium on Computer architecture, p.268-277, May 16-19, 1993, San Diego, California, United States
• 6: A. F. Benner , M. Ignatowski , J. A. Kash , D. M. Kuchta , M. B. Ritter, Exploitation of optical interconnects in future server architectures, IBM Journal of Research and Development, v.49 n.4/5, p.755-775, July 2005
• 7: [7] M. A. Blake, S. M. German, P. Mak, A. E. Seigler, and G. A. Huben. Bus protocol for a switchless distributed shared memory computer system. United States Patent #6,988,173 B2, International Business Machines Corporation, January 2006.
• 8: Shekhar Borkar, Low power design challenges for the decade (invited talk), Proceedings of the 2001 conference on Asia South Pacific design automation, p.293-296, January 2001, Yokohama, Japan [doi>10.1145/370155.370356]
• 9: [9] S. Y. Borkar, P. Dubey, K. C. Kahn, D. J. Kuck, H. Mulder, S. S. Pawlowski, and J. R. Rattner. Platform 2015: Intel processor and platform evolution for the next decade. Technical report, Intel White Paper, March 2005.
• 10: D. Burger , J. R. Goodman, Exploiting optical interconnects to eliminate serial bottlenecks, Proceedings of the 3rd Conference on Massively Parallel Processing Using Optical Interconnections, p.106, October 27-29, 1996
• 11: [11] R. T. Chang, N. Talwalkar, P. Yue, and S. S. Wong. Near speed-of-light signaling over on-chip electrical interconnects. IEEE Journal of Solid-State Circuits, 38(5):834-838, May 2003.
• 12: Alan Charlesworth, The sun fireplane system interconnect, Proceedings of the 2001 ACM/IEEE conference on Supercomputing (CDROM), p.7-7, November 10-16, 2001, Denver, Colorado [doi>10.1145/582034.582041]
• 13: [13] G. Chen, H. Chen, M. Haurylau, N. Nelson, D. Albonesi, P. M. Fauchet, and E. G. Friedman. Electrical and optical on-chip interconnects in scaled microprocessors. In International Symposium on Circuits and Systems, pages 2514-2517, Kobe, Japan, May 2005.
• 14: Guoqing Chen , Hui Chen , Mikhail Haurylau , Nicholas Nelson , Philippe M. Fauchet , Eby G. Friedman , David Albonesi, Predictions of CMOS compatible on-chip optical interconnect, Proceedings of the 2005 international workshop on System level interconnect prediction, April 02-03, 2005, San Francisco, California, USA [doi>10.1145/1053355.1053360]
• 15: [15] K.-N. Chen, M. J. Kobrinsky, B. C. Barnett, and R. Reif. Comparisons of conventional, 3-D, optical, and RF interconnects for on-chip clock distribution. IEEE Transactions on Electron Devices, 51(2):233-239, February 2004.
• 16: Xuning Chen , Li-Shiuan Peh , Gu-Yeon Wei , Yue-Kai Huang , Paul Prucnal, Exploring the Design Space of Power-Aware Opto-Electronic Networked Systems, Proceedings of the 11th International Symposium on High-Performance Computer Architecture, p.120-131, February 12-16, 2005 [doi>10.1109/HPCA.2005.15]
• 17: [17] J. Crow. Terabus Objectives and Challenges, C2COI Kickoff Meeting, http://www.darpa.mil/mto/c2oi/kick-off/Crow_Terabus.pdf, 2003.
• 18: David E. Culler , Anoop Gupta , Jaswinder Pal Singh, Parallel Computer Architecture: A Hardware/Software Approach, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1997
• 19: John D. Davis , James Laudon , Kunle Olukotun, Maximizing CMP Throughput with Mediocre Cores, Proceedings of the 14th International Conference on Parallel Architectures and Compilation Techniques, p.51-62, September 17-21, 2005 [doi>10.1109/PACT.2005.42]
• 20: [20] S. R. Deshpande. Method and apparatus for achieving correct order among bus memory transactions in a physically distributed SMP system. United States Patent #6,779,036, International Business Machines Corporation, August 2004.
• 21: [21] M. Haurylau, H. Chen, J. Zhang, G. Chen, N. A. Nelson, D. H. Albonesi, E. G. Friedman, and P. M. Fauchet. On-chip optical interconnect roadmap: Challenges and critical directions. In 2nd International Conference on Group IV Photonics, pages 17-19, Antwerp, Belgium, September 2005.
• 22: [22] R. Ho. On-Chip Wires: Scaling and Efficiency. Ph.D. dissertation, Dept. of Electrical Engineering, Stanford University, August 2003.
• 23: [23] R. Ho, W. Mai, and M. A. Horowitz. The future of wires. Proceedings of the IEEE, 89(4):490-504, April 2001.
• 24: [24] Intel White Paper. Next Leap in Microprocessor Architecture: Intel Core Duo, 2006.
• 25: [25] The ITRS Technology Working Groups, http://public.itrs.net. International Technology Roadmap for Semiconductors (ITRS) 2005 Edition.
• 26: [26] S. Fields J. M. Tendler, S. Dodson. POWER4 system microarchitecture. Technical report, IBM White Paper, October 2001.
• 27: Pawan Kapur , Gaurav Chandra , Krishna C. Saraswat, Power estimation in global interconnects and its reduction using a novel repeater optimization methodology, Proceedings of the 39th conference on Design automation, June 10-14, 2002, New Orleans, Louisiana, USA [doi>10.1145/513918.514035]
• 28: [28] P. Kapur and K. C. Saraswat. Comparisons between electrical and optical interconnects for on-chip signaling. In International Interconnect Technology Conference, pages 89-91, Burlingame, CA, June 2002.
• 29: [29] M. Kobrinsky, B. Block, J-F. Zheng, B. Barnett, E. Mohammed, M. Reshotko, F. Robertson, S. List, I. Young, and K. Cadien. On-chip optical interconnects. Intel Technology Journal, 08(02), May 2004.
• 30: Rakesh Kumar , Victor Zyuban , Dean M. Tullsen, Interconnections in Multi-Core Architectures: Understanding Mechanisms, Overheads and Scaling, Proceedings of the 32nd annual international symposium on Computer Architecture, p.408-419, June 04-08, 2005
• 31: James Laudon , Daniel Lenoski, The SGI Origin: a ccNUMA highly scalable server, Proceedings of the 24th annual international symposium on Computer architecture, p.241-251, June 01-04, 1997, Denver, Colorado, United States
• 32: D. Lenoski , J. Laudon , T. Joe , D. Nakahira , L. Stevens , A. Gupta , J. Hennessy, The DASH Prototype: Logic Overhead and Performance, IEEE Transactions on Parallel and Distributed Systems, v.4 n.1, p.41-61, January 1993 [doi>10.1109/71.205652]
• 33: [33] A. F. J. Levi. Fiber-to-the-Processor and Other Challenges for Photonics in Future Systems, http://asia.stanford.edu/events/Spring05/slides/050421- Levi.pdf, 2005.
• 34: [34] L. Liao, D. Samara-Rubio, M. Morse, A. Liu, D. Hodge, D. Rubin, U. Keil, and T. Franck. High-speed silicon Mach-Zehnder modulator. Optics Express, 13(8):3129-3135, April 2005.
• 35: [35] A. Liu, R. Jones, L. Liao, D. Samara-Rubio, D. Rubin, O. Cohen, R. Nicolaescu, and M. Paniccia. A high-speed silicon optical modulator based on a metal-oxide-semiconductor capacitor. Nature, 427:615-618, February 2004.
• 36: [36] A. Louri and A. K. Kodi. Parallel optical interconnection network for address transactions in large-scale cache coherent symmetric multiprocessors. IEEE Journal of Selected Topics on Quantum Electronics, 9(2):667-676, March-April 2003.
• 37: Ahmed Louri , Avinash Karanth Kodi, An Optical Interconnection Network and a Modified Snooping Protocol for the Design of Large-Scale Symmetric Multiprocessors (SMPs), IEEE Transactions on Parallel and Distributed Systems, v.15 n.12, p.1093-1104, December 2004 [doi>10.1109/TPDS.2004.75]
• 38: [38] D. A. Miller. Rationale and challenges for optical interconnects to electronic chips. Proceedings of the IEEE, 88(6):728- 749, June 2000.
• 39: [39] N. Nelson, G. Briggs, M. Haurylau, G. Chen, H. Chen, D. H. Albonesi, E. G. Friedman, and P. M. Fauchet. Alleviating thermal constraints while maintaining performance via silicon-based on-chip optical interconnects. In Workshop on Unique Chips and Systems, Austin, Texas, March 2005.
• 40: Ian O'Connor, Optical solutions for system-level interconnect, Proceedings of the 2004 international workshop on System level interconnect prediction, February 14-15, 2004, Paris, France [doi>10.1145/966747.966764]
• 41: [41] University of Illinois at Urbana-Champaign. http://sesc.sourceforge.net, 2005.
• 42: [42] A. Pappu and A. Apsel. Electrical isolation and fan-out in intra-chip optical interconnects. In International Symposium on Circuits and Systems, pages II-533-6 Vol. 2, Vancouver, Canada, May 2004.
• 43: [43] A. M. Pappu and A. B. Apsel. Analysis of intrachip electrical and optical fanout. Applied Optics, 44(30):6361-6372, October 2005.
• 44: [44] A. M. Pappu and A. B. Apsel. A low power, low delay TIA for on-chip applications. Conference on Lasers and Electro-Optics , 1:594-596, May 2005.
• 45: [45] P. Rabiei, W. H. Steier, C. Zhang, and L. R. Dalton. Polymer micro-ring filters and modulators. Journal of Lightwave Technology, 20(11):1968-1975, November 2002.
• 46: Arifur Rahman , Rafael Reif, System-level performance evaluation of three-dimensional integrated circuits, IEEE Transactions on Very Large Scale Integration (VLSI) Systems, v.8 n.6, p.671-678, Dec. 2000 [doi>10.1109/92.902261]
• 47: Harshit Shah , Pun Shiu , Brian Bell , Mamie Aldredge , Namarata Sopory , Jeff Davis, Repeater insertion and wire sizing optimization for throughput-centric VLSI global interconnects, Proceedings of the 2002 IEEE/ACM international conference on Computer-aided design, p.280-284, November 10-14, 2002, San Jose, California [doi>10.1145/774572.774614]
• 48: [48] R. A. Soref and B. R. Bennett. Electrooptical effects in silicon. IEEE Journal on Quantum Electronics, 23(1):123-129, January 1987.
• 49: Shukri J. Souri , Kaustav Banerjee , Amit Mehrotra , Krishna C. Saraswat, Multiple Si layer ICs: motivation, performance analysis, and design implications, Proceedings of the 37th conference on Design automation, p.213-220, June 05-09, 2000, Los Angeles, California, United States [doi>10.1145/337292.337394]
• 50: Karin Strauss , Xiaowei Shen , Josep Torrellas, Flexible Snooping: Adaptive Forwarding and Filtering of Snoops in Embedded-Ring Multiprocessors, Proceedings of the 33rd annual international symposium on Computer Architecture, p.327-338, June 17-21, 2006
• 51: [51] D. Tarjan, S. Thoziyoor, and N. P. Jouppi. Cacti 4.0. Technical Report HPL-2006-86, HP Laboratories Palo Alto, June 2006. http://quid.hpl.hp.com:9081/cacti/.
• 52: [52] J. Tatum. VCSELs for 10 GB/s optical interconnects. In IEEE Emerging Technologies Symposium on BroadBand Communications for the Internet Era, pages 58-61, Richardson, TX, September 2001.
• 53: [53] J. M. Tendler, J. S. Dodson, J. S. Fields, H. Le, and B. Sinharoy. POWER4 system microarchitecture. IBM Journal of Research and Development, 46(1):5-25, January 2002.
• 54: Hang-Sheng Wang , Xinping Zhu , Li-Shiuan Peh , Sharad Malik, Orion: a power-performance simulator for interconnection networks, Proceedings of the 35th annual ACM/IEEE international symposium on Microarchitecture, November 18-22, 2002, Istanbul, Turkey
• 55: Brian Webb , Ahmed Louri, A Class of Highly Scalable Optical Crossbar-Connected Interconnection Networks (SOCNs) for Parallel Computing Systems, IEEE Transactions on Parallel and Distributed Systems, v.11 n.5, p.444-458, May 2000 [doi>10.1109/71.852398]
• 56: [56] S. M. Weiss, M. Molinari, and P. M. Fauchet. Temperature stability for silicon-based photonic band-gap structures. Applied Physics Letters, 83(10):1980-1982, September 2003.
• 57: Steven Cameron Woo , Moriyoshi Ohara , Evan Torrie , Jaswinder Pal Singh , Anoop Gupta, The SPLASH-2 programs: characterization and methodological considerations, Proceedings of the 22nd annual international symposium on Computer architecture, p.24-36, June 22-24, 1995, S. Margherita Ligure, Italy
• 58: [58] T. K. Woodward and A. V. Krishnamoorthy. 1-Gb/s integrated optical detectors and receivers in commercial CMOS technologies. IEEE Journal of Selected Topics on Quantum Electronics, 5(2):146-156, March-April 1999.
• 59: [59] T. Yin, A. M. Pappu, and A. B. Apsel. Low-cost, high-efficiency, and high-speed SiGe phototransistors in commercial BiCMOS. IEEE Photonics Technology Letters, 18(1):55- 57, January 2006.
• 60: [60] I. Young. Intel introduces chip-to-chip optical I/O interconnect prototype. Technology@Intel Magazine, pages 3-7, April 2004.

INDEX TERMS

Primary Classification:
  B. Hardware
  B.4 INPUT/OUTPUT AND DATA COMMUNICATIONS
      B.4.3 Interconnections (subsystems)
          Subjects: Topology (e.g., bus, point-to-point)

Additional Classification:
  B. Hardware
  B.8 Performance and Reliability
      B.8.2 Performance Analysis and Design Aids

  C. Computer Systems Organization
  C.1 PROCESSOR ARCHITECTURES
      C.1.2 Multiple Data Stream Architectures (Multiprocessors)

General Terms:
Design, Performance