Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

We explore the nature of traffic in data centers, designed to support the mining of massive data sets. We instrument the servers to collect socket-level logs, with negligible performance impact. In a 1500 server operational cluster, we thus amass roughly a petabyte of measurements over two months, from which we obtain and report detailed views of traffic and congestion conditions and patterns. We further consider whether traffic matrices in the cluster might be obtained instead via tomographic inference from coarser-grained counter data.

REFERENCES

• 1: AmazonWeb Services. http://aws.amazon.com.
• 2: EventTracing forWindows. http://msdn.microso.com/en-us/library/ms751538.aspx.
• 3: Google app engine. http://code.google.com/appengine/.
• 4: Hadoop distributed filesystem. http://hadoop.apache.org.
• 5: Windows Azure. http://www.microso.com/azure/.
• 6: Urs Hoelzle , Luiz Andre Barroso, The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines, Morgan and Claypool Publishers, 2009
• 7: Theophilus Benson , Ashok Anand , Aditya Akella , Ming Zhang, Understanding data center traffic characteristics, Proceedings of the 1st ACM workshop on Research on enterprise networking, August 21-21, 2009, Barcelona, Spain [doi>10.1145/1592681.1592692]
• 8: Ronnie Chaiken , Bob Jenkins , Per-Åke Larson , Bill Ramsey , Darren Shakib , Simon Weaver , Jingren Zhou, SCOPE: easy and efficient parallel processing of massive data sets, Proceedings of the VLDB Endowment, v.1 n.2, August 2008 [doi>10.1145/1454159.1454166]
• 9: Fay Chang , Jeffrey Dean , Sanjay Ghemawat , Wilson C. Hsieh , Deborah A. Wallach , Mike Burrows , Tushar Chandra , Andrew Fikes , Robert E. Gruber, Bigtable: a distributed storage system for structured data, Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation, p.15-15, November 06-08, 2006, Seattle, WA
• 10: Yanpei Chen , Rean Griffith , Junda Liu , Randy H. Katz , Anthony D. Joseph, Understanding TCP incast throughput collapse in datacenter networks, Proceedings of the 1st ACM workshop on Research on enterprise networking, August 21-21, 2009, Barcelona, Spain [doi>10.1145/1592681.1592693]
• 11: Cisco Guard DDoS Mitigation Appliance. http://www.cisco.com/en/US/products/ps5888/.
• 12: Cisco Nexus 7000 Series Switches. http://www.cisco.com/en/US/products/ps9402/.
• 13: Chuck Cranor , Theodore Johnson , Oliver Spataschek , Vladislav Shkapenyuk, Gigascope: a stream database for network applications, Proceedings of the 2003 ACM SIGMOD international conference on Management of data, June 09-12, 2003, San Diego, California [doi>10.1145/872757.872838]
• 14: Jeffrey Dean , Sanjay Ghemawat, MapReduce: simplified data processing on large clusters, Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation, p.10-10, December 06-08, 2004, San Francisco, CA
• 15: Nick Duffield , Carsten Lund , Mikkel Thorup, Estimating flow distributions from sampled flow statistics, Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications, August 25-29, 2003, Karlsruhe, Germany [doi>10.1145/863955.863992]
• 16: Cristian Estan , Ken Keys , David Moore , George Varghese, Building a better NetFlow, Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications, August 30-September 03, 2004, Portland, Oregon, USA
• 17: Sanjay Ghemawat , Howard Gobioff , Shun-Tak Leung, The Google file system, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 18: Albert Greenberg , James R. Hamilton , Navendu Jain , Srikanth Kandula , Changhoon Kim , Parantap Lahiri , David A. Maltz , Parveen Patel , Sudipta Sengupta, VL2: a scalable and flexible data center network, Proceedings of the ACM SIGCOMM 2009 conference on Data communication, August 16-21, 2009, Barcelona, Spain
• 19: Saikat Guha , Jaideep Chandrashekar , Nina Taft , Konstantina Papagiannaki, How healthy are today's enterprise networks?, Proceedings of the 8th ACM SIGCOMM conference on Internet measurement, October 20-22, 2008, Vouliagmeni, Greece [doi>10.1145/1452520.1452538]
• 20: Anders Gunnar , Mikael Johansson , Thomas Telkamp, Traffic matrix estimation on a large IP backbone: a comparison on real data, Proceedings of the 4th ACM SIGCOMM conference on Internet measurement, October 25-27, 2004, Taormina, Sicily, Italy [doi>10.1145/1028788.1028807]
• 21: L. Huang, X. Nguyen, M. Garofalakis, J. Hellerstein, M. Jordan, M. Joseph, and N. Taft. Communication-Efficient Online Detection of Network-Wide Anomalies. In INFOCOM, 2007.
• 22: IETF Working Group IP Flow Information Export (ipfix). http://www.ietf.org/html.charters/ipfix-charter.html.
• 23: Michael Isard , Mihai Budiu , Yuan Yu , Andrew Birrell , Dennis Fetterly, Dryad: distributed data-parallel programs from sequential building blocks, Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007, March 21-23, 2007, Lisbon, Portugal
• 24: Thomas Karagiannis , Richard Mortier , Antony Rowstron, Network exception handlers: host-network control in enterprise networks, Proceedings of the ACM SIGCOMM 2008 conference on Data communication, August 17-22, 2008, Seattle, WA, USA
• 25: M. Kodialam, T. V. Lakshman, and S. Sengupta. Efficient and Robust Routing of Highly Variable Traffic. In HotNets, 2004.
• 26: Ramana Rao Kompella , Cristian Estan, The power of slicing in internet flow measurement, Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement, p.9-9, October 19-21, 2005, Berkeley, CA
• 27: Christopher Olston , Benjamin Reed , Utkarsh Srivastava , Ravi Kumar , Andrew Tomkins, Pig latin: a not-so-foreign language for data processing, Proceedings of the 2008 ACM SIGMOD international conference on Management of data, June 09-12, 2008, Vancouver, Canada [doi>10.1145/1376616.1376726]
• 28: Ruoming Pang , Mark Allman , Mike Bennett , Jason Lee , Vern Paxson , Brian Tierney, A first look at modern enterprise traffic, Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement, p.2-2, October 19-21, 2005, Berkeley, CA
• 29: IETF Packet Sampling (ActiveWG). http://tools.ietf.org/wg/psamp/.
• 30: S. Kandula and D. Katabi and S. Sinha and A. Berger. Dynamic Load Balancing Without Packet Reordering. In CCR, 2006.
• 31: sFlow.org. Making the network visible. http://www.sflow.org.
• 32: Augustin Soule , Anukool Lakhina , Nina Taft , Konstantina Papagiannaki , Kave Salamatian , Antonio Nucci , Mark Crovella , Christophe Diot, Traffic matrices: balancing measurements, inference and modeling, Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, June 06-10, 2005, Banff, Alberta, Canada
• 33: Vijay Vasudevan , Amar Phanishayee , Hiral Shah , Elie Krevat , David G. Andersen , Gregory R. Ganger , Garth A. Gibson , Brian Mueller, Safe and effective fine-grained TCP retransmissions for datacenter communication, Proceedings of the ACM SIGCOMM 2009 conference on Data communication, August 16-21, 2009, Barcelona, Spain
• 34: Yin Zhang , Zihui Ge , Albert Greenberg , Matthew Roughan, Network anomography, Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement, p.30-30, October 19-21, 2005, Berkeley, CA
• 35: Yin Zhang , Matthew Roughan , Nick Duffield , Albert Greenberg, Fast accurate computation of large-scale IP traffic matrices from link loads, Proceedings of the 2003 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, June 11-14, 2003, San Diego, CA, USA
• 36: R. Zhang-Shen and N. McKeown. Designing a Predictable Internet Backbone Network. In HotNets, 2004.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.4 Distributed Systems
          Subjects: Distributed applications

Additional Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Performance attributes

General Terms:
Design, Experimentation, Measurement, Performance

Keywords:
characterization, data center traffic, models, tomography