Firefly: illuminating future network-on-chip with nanophotonics

ABSTRACT

Future many-core processors will require high-performance yet energy-efficient on-chip networks to provide a communication substrate for the increasing number of cores. Recent advances in silicon nanophotonics create new opportunities for on-chip networks. To efficiently exploit the benefits of nanophotonics, we propose Firefly - a hybrid, hierarchical network architecture. Firefly consists of clusters of nodes that are connected using conventional, electrical signaling while the inter-cluster communication is done using nanophotonics - exploiting the benefits of electrical signaling for short, local communication while nanophotonics is used only for global communication to realize an efficient on-chip network. Crossbar architecture is used for inter-cluster communication. However, to avoid global arbitration, the crossbar is partitioned into multiple, logical crossbars and their arbitration is localized. Our evaluations show that Firefly improves the performance by up to 57% compared to an all-electrical concentrated mesh (CMESH) topology on adversarial traffic patterns and up to 54% compared to an all-optical crossbar (OP XBAR) on traffic patterns with locality. If the energy-delay-product is compared, Firefly improves the efficiency of the on-chip network by up to 51% and 38% compared to CMESH and OP XBAR, respectively.

REFERENCES

• 1: A. Al-Azzawi. Photonics: Principles and Practices. CRC Press, 2007.
• 2: V. Almeida, C. Barrios, R. Panepucci, M. Lipson, M. Foster, D. Ouzounov, and A. Gaeta. All-optical switching on a silicon chip. Optics Letters, 29:2867--2869, 2004.
• 3: James Balfour , William J. Dally, Design tradeoffs for tiled CMP on-chip networks, Proceedings of the 20th annual international conference on Supercomputing, June 28-July 01, 2006, Cairns, Queensland, Australia [doi>10.1145/1183401.1183430]
• 4: C. A. Barrios, V. R. Almeida, and M. Lipson. Low-power-consumption short-length and high-modulation-depth silicon electro-optic modulator. Journal of Lightwave Technology, 21(4):1089--1098, 2003.
• 5: Christopher Batten , Ajay Joshi , Jason Orcutt , Anatoly Khilo , Benjamin Moss , Charles Holzwarth , Milos Popovic , Hanqing Li , Henry Smith , Judy Hoyt , Franz Kartner , Rajeev Ram , Vladimir Stojanovic , Krste Asanovic, Building Manycore Processor-to-DRAM Networks with Monolithic Silicon Photonics, Proceedings of the 2008 16th IEEE Symposium on High Performance Interconnects, p.21-30, August 26-28, 2008 [doi>10.1109/HOTI.2008.11]
• 6: S. Bell, B. Edwards, J. Amann, R. Conlin, K. Joyce, V. Leung, J. Mackay, M. Reif, L. Bao, J. Brown, M. Mattina, C.-C. Miao, C. Ramey, D. Wentzlaff, W. Anderson, E. Berger, N. Fairbanks, D. Khan, F. Montenegro, J. Stickney, and J. Zook. Tile64 processor: A 64-core soc with mesh interconnect. In Solid-State Circuits Conference, 2008. ISSCC 2008. Digest of Technical Papers. IEEE International, pages 88--598, 2008.
• 7: N. Binkert. Personal communication, Aug. 2008.
• 8: Roger D. Chamberlain , Mark A. Franklin , Ch'ng Shi Baw, Gemini: An Optical Interconnection Network for Parallel Processing, IEEE Transactions on Parallel and Distributed Systems, v.13 n.10, p.1038-1055, October 2002 [doi>10.1109/TPDS.2002.1041880]
• 9: M. Chang, J. Cong, A. Kaplan, M. Naik, G. Reinman, E. Socher, and S.-W. Tam. CMP network-on-chip overlaid with multi-band rf-interconnect. In International Symposium on High-Performance Computer Architecture (HPCA), pages 191--202, Feb. 2008.
• 10: Guoqing Chen , Hui Chen , Mikhail Haurylau , Nicholas Nelson , Philippe M. Fauchet , Eby G. Friedman , David Albonesi, Predictions of CMOS compatible on-chip optical interconnect, Proceedings of the 2005 international workshop on System level interconnect prediction, April 02-03, 2005, San Francisco, California, USA [doi>10.1145/1053355.1053360]
• 11: William Dally , Brian Towles, Principles and Practices of Interconnection Networks, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2003
• 12: William J. Dally , Brian Towles, Route packets, not wires: on-chip inteconnection networks, Proceedings of the 38th annual Design Automation Conference, p.684-689, June 2001, Las Vegas, Nevada, United States [doi>10.1145/378239.379048]
• 13: R. Das, S. Eachempati, A. Mishra, V. Narayanan, and C. Das. Design and evaluation of a hierarchical on-chip interconnect for next-generation CMPs. In International Symposium on High-Performance Computer Architecture (HPCA), pages 175--186, Raleigh, NC, USA, Feb. 2009.
• 14: P. Gratz, C. Kim, R. McDonald, S. Keckler, and D. Burger. Implementation and evaluation of on-chip network architectures. In International Conference on Computer Design (ICCD), pages 477--484, San Jose, CA, 2006.
• 15: A. Gubenko, I. Krestnikov, D. Livshtis, S. Mikhrin, A. Kovsh, L. West, C. Bornholdt, N. Grote, and A. Zhukov. Error-free 10 gbit/s transmission using individual fabry-perot modes of low-noise quantum-dot laser. Electronic Letters, 43(25):1430--1431, 2007.
• 16: Nan Jiang , John Kim , William J. Dally, Indirect adaptive routing on large scale interconnection networks, Proceedings of the 36th annual international symposium on Computer architecture, June 20-24, 2009, Austin, TX, USA
• 17: A. Jose and K. Shepard. Distributed loss-compensation techniques for energy-efficient low-latency on-chip communication. Solid-State Circuits, IEEE Journal of, 42(6):1415--1424, June 2007.
• 18: P. Kapur and K. C. Saraswat. Comparisons between electrical and optical interconnects for on-chip signaling. In International Interconnect Technology Conference, pages 89--91, Burlingame, CA, Jun. 2002.
• 19: P. Kermani and L. Kleinrock. Virtual cut-through: A new computer communication switching technique. Computer Networks, 3:267--286, 1979.
• 20: John Kim , James Balfour , William Dally, Flattened Butterfly Topology for On-Chip Networks, Proceedings of the 40th Annual IEEE/ACM International Symposium on Microarchitecture, p.172-182, December 01-05, 2007 [doi>10.1109/MICRO.2007.15]
• 21: John Kim , Wiliam J. Dally , Steve Scott , Dennis Abts, Technology-Driven, Highly-Scalable Dragonfly Topology, Proceedings of the 35th International Symposium on Computer Architecture, p.77-88, June 21-25, 2008
• 22: John Kim , William Dally , Steve Scott , Dennis Abts, Cost-Efficient Dragonfly Topology for Large-Scale Systems, IEEE Micro, v.29 n.1, p.33-40, January 2009 [doi>10.1109/MM.2009.5]
• 23: John Kim , William J. Dally , Brian Towles , Amit K. Gupta, Microarchitecture of a High-Radix Router, Proceedings of the 32nd annual international symposium on Computer Architecture, p.420-431, June 04-08, 2005
• 24: Nevin Kirman , Meyrem Kirman , Rajeev K. Dokania , Jose F. Martinez , Alyssa B. Apsel , Matthew A. Watkins , David H. Albonesi, Leveraging Optical Technology in Future Bus-based Chip Multiprocessors, Proceedings of the 39th Annual IEEE/ACM International Symposium on Microarchitecture, p.492-503, December 09-13, 2006 [doi>10.1109/MICRO.2006.28]
• 25: A. Kovsh, I. Krestnikov, D. Livshits, S. Mikhrin, J. Weimert, and A. Zhukov. Quantum dot laser with 75nm broad spectrum of emission. Optics Letters, 32(7):793--795, 2007.
• 26: Tushar Krishna , Amit Kumar , Patrick Chiang , Mattan Erez , Li-Shiuan Peh, NoC with Near-Ideal Express Virtual Channels Using Global-Line Communication, Proceedings of the 2008 16th IEEE Symposium on High Performance Interconnects, p.11-20, August 26-28, 2008 [doi>10.1109/HOTI.2008.22]
• 27: A. Kumar, P. Kundu, A. P. Singh, L.-S. Peh, and N. K. Jha. A 4.6tbits/s 3.6ghz single-cycle NoC router with a novel switch allocator in 65nm CMOS. In International Conference on Computer Design (ICCD), pages 63--70, Lake Tahoe, CA, 2007.
• 28: Prabhat Kumar , Yan Pan , John Kim , Gokhan Memik , Alok Choudhary, Exploring concentration and channel slicing in on-chip network router, Proceedings of the 2009 3rd ACM/IEEE International Symposium on Networks-on-Chip, p.276-285, May 10-13, 2009 [doi>10.1109/NOCS.2009.5071477]
• 29: Ahmed Louri , Avinash Karanth Kodi, An Optical Interconnection Network and a Modified Snooping Protocol for the Design of Large-Scale Symmetric Multiprocessors (SMPs), IEEE Transactions on Parallel and Distributed Systems, v.15 n.12, p.1093-1104, December 2004 [doi>10.1109/TPDS.2004.75]
• 30: R. Narayanan, B. Ozisikyilmaz, J. Zambreno, G. Memik, and A. Choudhary. Minebench: A benchmark suite for data mining workloads. In IEEE International Symposium on Workload Characterization (IISCW), pages 182--188, San Jose, CA, 2006.
• 31: O.I.Dosunmu, M. K. Emsley, M. S. Unlu, D. DCannon, and L. C. Kimerling. High speed resonant cavity enhanced ge photodetectors on si reflecting substrates for 1550 nm operation. In IEEE International Topical Meeting on Microwave Photonics, 2004., pages 266--268, Ogunquit, ME, 2004.
• 32: L. Pavesi and D. J. Lockwood. Silicon photonics, 2004.
• 33: M. Petracca, K. Bergman, and L. Carloni. Photonic networks-on-chip: Opportunities and challenges. pages 2789--2792, Seattle, WA, May 2008.
• 34: T. Pinkston. Design considerations for optical interconnects in parallel computers. In Proc. of the First International Workshop on Massively Parallel Processing Using Optical Interconnections, pages 306--322, Cancun, Mexico, Apr. 1994.
• 35: Assaf Shacham , Keren Bergman , Luca P. Carloni, The case for low-power photonic networks on chip, Proceedings of the 44th annual Design Automation Conference, June 04-08, 2007, San Diego, California [doi>10.1145/1278480.1278513]
• 36: J. Tatum. Vcsels for 10 gb/s optical interconnects. In IEEE Emerging Technologies Symposium on BroadBand Communications for the Internet Era, pages 58--61, Richardson, TX, 2001.
• 37: S. R. Vangal, J. Howard, G. Ruhl, S. Dighe, H. Wilson, J. Tschanz, D. Finan, A. Singh, T. Jacob, S. Jain, V. Erraguntla, C. Roberts, Y. Hoskote, N. Borkar, and S. Borkar. An 80-Tile sub-100-w teraflops processor in 65-nm cmos. Solid-State Circuits, IEEE Journal of, 43(1):29--41, 2008.
• 38: Dana Vantrease , Robert Schreiber , Matteo Monchiero , Moray McLaren , Norman P. Jouppi , Marco Fiorentino , Al Davis , Nathan Binkert , Raymond G. Beausoleil , Jung Ho Ahn, Corona: System Implications of Emerging Nanophotonic Technology, Proceedings of the 35th International Symposium on Computer Architecture, p.153-164, June 21-25, 2008
• 39: Y. Vlasov and S. McNab. Losses in single-mode silicon-on-insulator strip waveguides and bends. Optics Express, 12(8):1622--1631, 2004.
• 40: Steven Cameron Woo , Moriyoshi Ohara , Evan Torrie , Jaswinder Pal Singh , Anoop Gupta, The SPLASH-2 programs: characterization and methodological considerations, Proceedings of the 22nd annual international symposium on Computer architecture, p.24-36, June 22-24, 1995, S. Margherita Ligure, Italy
• 41: Q. Xu, S. Manipatruni, B. Schmidt, J. Shakya, and M. Lipson. 12.5 gbit/s carrier-injection-based silicon micro-ring silicon modulators. Opt. Express, 15(2):430--436, Jan. 2007.
• 42: T. Yin, R. Cohen, M. M. Morse, G. Sarid, Y. Chetrit, D. Rubin, and M. J. Paniccia. 40gb/s ge-on-soi waveguide photodetectors by selective ge growth. In Conference on Optical Fiber communication/National Fiber Optic Engineers Conference (OFC/NFOEC), pages 24--28, San Diego, CA, 2008.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.1 PROCESSOR ARCHITECTURES
      C.1.2 Multiple Data Stream Architectures (Multiprocessors)
          Subjects: Interconnection architectures (e.g., common bus, multiport memory, crossbar switch)

Additional Classification:
  B. Hardware
  B.4 INPUT/OUTPUT AND DATA COMMUNICATIONS
      B.4.3 Interconnections (subsystems)
          Subjects: Topology (e.g., bus, point-to-point)

General Terms:
Design, Performance

Keywords:
hierarchical network, interconnection networks, nanophotonics, topology