BCube: a high performance, server-centric network architecture for modular data centers

ABSTRACT

This paper presents BCube, a new network architecture specifically designed for shipping-container based, modular data centers. At the core of the BCube architecture is its server-centric network structure, where servers with multiple network ports connect to multiple layers of COTS (commodity off-the-shelf) mini-switches. Servers act as not only end hosts, but also relay nodes for each other. BCube supports various bandwidth-intensive applications by speeding-up one-to-one, one-to-several, and one-to-all traffic patterns, and by providing high network capacity for all-to-all traffic.

BCube exhibits graceful performance degradation as the server and/or switch failure rate increases. This property is of special importance for shipping-container data centers, since once the container is sealed and operational, it becomes very difficult to repair or replace its components.

Our implementation experiences show that BCube can be seamlessly integrated with the TCP/IP protocol stack and BCube packet forwarding can be efficiently implemented in both hardware and software. Experiments in our testbed demonstrate that BCube is fault tolerant and load balancing and it significantly accelerates representative bandwidth-intensive applications.

REFERENCES

• 1: Mohammad Al-Fares , Alexander Loukissas , Amin Vahdat, A scalable, commodity data center network architecture, Proceedings of the ACM SIGCOMM 2008 conference on Data communication, August 17-22, 2008, Seattle, WA, USA
• 2: Luiz André Barroso , Jeffrey Dean , Urs Hölzle, Web Search for a Planet: The Google Cluster Architecture, IEEE Micro, v.23 n.2, p.22-28, March 2003 [doi>10.1109/MM.2003.1196112]
• 3: L. N. Bhuyan , D. P. Agrawal, Generalized Hypercube and Hyperbus Structures for a Computer Network, IEEE Transactions on Computers, v.33 n.4, p.323-333, April 1984 [doi>10.1109/TC.1984.1676437]
• 4: D. Borthakur. The Hadoop Distributed File System: Architecture and Design. http://hadoop.apache.org/core/docs/current/hdfs_design.pdf.
• 5: CloudStore. Higher Performance Scalable Storage. http://kosmosfs.sourceforge.net/.
• 6: Jeffrey Dean , Sanjay Ghemawat, MapReduce: simplified data processing on large clusters, Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation, p.10-10, December 06-08, 2004, San Francisco, CA
• 7: Albert Greenberg , Parantap Lahiri , David A. Maltz , Parveen Patel , Sudipta Sengupta, Towards a next generation data center architecture: scalability and commoditization, Proceedings of the ACM workshop on Programmable routers for extensible services of tomorrow, August 22-22, 2008, Seattle, WA, USA [doi>10.1145/1397718.1397732]
• 8: Albert Greenberg , James R. Hamilton , Navendu Jain , Srikanth Kandula , Changhoon Kim , Parantap Lahiri , David A. Maltz , Parveen Patel , Sudipta Sengupta, VL2: a scalable and flexible data center network, Proceedings of the ACM SIGCOMM 2009 conference on Data communication, August 16-21, 2009, Barcelona, Spain
• 9: Chuanxiong Guo , Haitao Wu , Kun Tan , Lei Shi , Yongguang Zhang , Songwu Lu, Dcell: a scalable and fault-tolerant network structure for data centers, Proceedings of the ACM SIGCOMM 2008 conference on Data communication, August 17-22, 2008, Seattle, WA, USA
• 10: G. Lu et al. CAFE: A Configurable pAcket Forwarding Engine for Data Center Networks. In SIGCOMM PRESTO Workshop, Aug 2009.
• 11: Jose Duato , Sudhakar Yalamanchili , Ni Lionel, Interconnection Networks: An Engineering Approach, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2002
• 12: Sanjay Ghemawat , Howard Gobioff , Shun-Tak Leung, The Google file system, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 13: J. Hamilton. An Architecture for Modular Data Centers. In 3rd CIDR, Jan 2007.
• 14: J. Hamilton. Cooperative Expandable Micro-Slice Servers (CEMS). In 4th CIDR, Jan 2009.
• 15: J. Hamilton. Private communication, 2009.
• 16: IBM. Scalable Modular Data Center. http://www-935.ibm.com/services/us/its/pdf/smdc-eb-sfe03001-usen-00-022708.p%df.
• 17: Michael Isard , Mihai Budiu , Yuan Yu , Andrew Birrell , Dennis Fetterly, Dryad: distributed data-parallel programs from sequential building blocks, Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007, March 21-23, 2007, Lisbon, Portugal
• 18: F. Thomson Leighton, Introduction to parallel algorithms and architectures: array, trees, hypercubes, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1991
• 19: Charles E. Leiserson, Fat-trees: universal networks for hardware-efficient supercomputing, IEEE Transactions on Computers, v.34 n.10, p.892-901, Oct. 1985
• 20: John T. Moy, OSPF: Anatomy of an Internet Routing Protocol, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1998
• 21: Jad Naous , Glen Gibb , Sara Bolouki , Nick McKeown, NetFPGA: reusable router architecture for experimental research, Proceedings of the ACM workshop on Programmable routers for extensible services of tomorrow, August 22-22, 2008, Seattle, WA, USA [doi>10.1145/1397718.1397720]
• 22: Silicom. Gigabit Ethnet Server Adapters. http://www.silicom--usa.com/default.asp?contentID=711.
• 23: Rackable Systems. ICE Cube Modular Data Center. http://www.rackable.com/products/icecube.aspx.
• 24: Verari Systems. The Verari FOREST Container Solution: The Answer to Consolidation. http://www.verari.com/forest_spec.asp.
• 25: M. Waldrop. Data Center in a Box. Scientific American, July 2007.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.1 Network Architecture and Design
          Subjects: Network topology

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.1 Network Architecture and Design
          Subjects: Packet-switching networks

General Terms:
Algorithms, Design

Keywords:
modular data center, multi-path, server-centric network