Experience in measuring backbone traffic variability: models, metrics, measurements and meaning

ABSTRACT

Understanding the variability of Internet traffic in backbone networks is essential to better plan and manage existing networks, as well as to design next generation networks. However, most traffic analyses that might be used to approach this problem are based on detailed packet or flow level measurements, which are usually not available throughout a large network. As a result there is a poor understanding of backbone traffic variability, and its impact on network operations (e.g. on capacity planning or traffic engineering).This paper introduces a metric for measuring backbone traffic variability that is grounded on simple but powerful traffic theory. What sets this metric apart, however, is that we present a method for making practical measurements of the metric using widely available SNMP traffic measurements. Furthermore, we use a novel method to overcome the major limitation of SNMP measurements -- that they only provide link statistics. The method, based on a "gravity model", derives an approximate traffic matrix from the SNMP data. In addition to simulations, we use more than 1 year's worth of SNMP data from an operational IP network of about 1000 nodes to test our methods. We also delve into the degree and sources of variability in real backbone traffic, providing insight into the true nature of traffic variability.

REFERENCES

• 1: Will E. Leland , Murad S. Taqqu , Walter Willinger , Daniel V. Wilson, On the self-similar nature of Ethernet traffic (extended version), IEEE/ACM Transactions on Networking (TON), v.2 n.1, p.1-15, Feb. 1994 [doi>10.1109/90.282603]
• 2: Walter Willinger , Vern Paxson , Murad S. Taqqu, Self-similarity and heavy tails: structural modeling of network traffic, A practical guide to heavy tails: statistical techniques and applications, Birkhauser Boston Inc., Cambridge, MA, 1998
• 3: Vern Paxson, Empirically derived analytic models of wide-area TCP connections, IEEE/ACM Transactions on Networking (TON), v.2 n.4, p.316-336, Aug. 1994 [doi>10.1109/90.330413]
• 4: J. Cao, W. S. Cleveland, D. Lin, and D. X. Sun, "The effect of statistical multiplexing on internet packet traffic: Theory and empirical study," tech. rep., Bell Labs, 2001. Available at http://cm.bell-labs.com/cm/ms/departments/sia/wsc/publish.html.
• 5: B.S. Arnaud, "Current optical network designs may be flawed," Optical Networks Magazine, voL 2, March/April 2001.
• 6: J. Mooney, "Gaining the edge in flexible metro service provisioning;" Lightwave, February 2001. Available at http://www.sycamorenet.com/corporate/articles/reprintedarticle.asp?comm%and=live&news_item id=437.
• 7: Anja Feldmann , Albert Greenberg , Carsten Lund , Nick Reingold , Jennifer Rexford , Fred True, Deriving traffic demands for operational IP networks: methodology and experience, IEEE/ACM Transactions on Networking (TON), v.9 n.3, p.265-280, June 2001 [doi>10.1109/90.929850]
• 8: J.Y. Wei, "IP over WDM network traffic engineering approaches," in invited talk Optical Fiber Commun. Conf. (OFC), 2002.
• 9: D. Awduche and Y. Rekhter, "Multiprotocol Iambda switching: combining mpls traffic engineering control with optical crossconnects;" IEEE Communications Magazine, vol. 39, pp. 111--116, March 2001.
• 10: Dhritiman Banerjee , Biswanath Mukherjee, Wavelength-Routed Optical Networks: Linear Formulation, Resource Budgeting Tradeoffs, and a Reconfiguration Study, Proceedings of the INFOCOM '97. Sixteenth Annual Joint Conference of the IEEE Computer and Communications Societies. Driving the Information Revolution, p.269, April 09-11, 1997
• 11: I. Norros, "A storage mode! with self-similar input;" Queueing Systems, voL 16, pp. 387--396, 1994.
• 12: Y. Vardi, "Network tomography," Journal of the American Statistical Association, March 1996.
• 13: C. Tebaldi and M. West, "Bayesian inference on network traffic;" Journal of the American Statistical Association, June t998.
• 14: J. Cao, D. Davis, S. V. Wiel, and B. Yu, "Time-varying network tomography;' Journal of the American Statistical Association, December 2000.
• 15: N. G. Duffield , Matthias Grossglauser, Trajectory sampling for direct traffic observation, IEEE/ACM Transactions on Networking (TON), v.9 n.3, p.280-292, June 2001 [doi>10.1109/90.929851]
• 16: J. Kowalski and B. Warfield, "Modeling traffic demand between nodes in a telecommunications network," in ATNAC'95, t995.
• 17: P. Pongpaibool, R. Doverspike, M. Roughan, and J. Gottlieb, "Handling ip traffic surges via optical layer reconfiguration;" in to appear in OFC, 2002.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Measurement techniques

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.3 Network Operations
          Subjects: Network management
      C.2.5 Local and Wide-Area Networks
          Subjects: Internet (e.g., TCP/IP)

General Terms:
Management, Measurement, Performance