Rapid model parameterization from traffic measurements

ABSTRACT

The utility of simulations and analysis heavily relies on good models of network traffic. While network traffic constantly is changing over time, existing approaches typically take years from collecting trace, analyzing the data to finally generating and implementing models. In this paper, we describe approaches and tools that support rapid parameterization of traffic models from live network measurements. Rather than treating measured traffic as a time-series of statistics, we utilize the traces to estimate end-user behavior and network conditions to generate application-level simulation models. We also show multi-scaling analytic techniques are helpful for debugging and validating the model. To demonstrate our approaches, we develop structural source-level models for web and FTP traffic and evaluate their accuracy by comparing the outputs of simulation against the original trace. We also compare our work with existing traffic generation tools and show our approach is more flexible in capturing the heterogeneity of traffic. Finally, we automate and integrate the process from trace analysis to model validation for easy model parameterization from new data.

REFERENCES

• 1: Abry, P. and Veitch, D. 1998. Wavelet analysis of long-range-dependent traffic. IEEE Trans. Inform. Theory 44, 1, 2--15.
• 2: Asaba, T., Claffy, K., Nakamura, O., and Murai, J. 1992. An analysis of international academic research network traffic between japan and other nations. In Inet '92. 431--440.
• 3: Balakrishnan, H., Padmanabhan, V. N., and Katz, R. H. 1998. Tcp behavior of a busy internet server: Analysis and improvements. In Proceedings of the IEEE Infocom. IEEE, San Francisco, CA, USA.
• 4: Hari Balakrishnan , Mark Stemm , Srinivasan Seshan , Randy H. Katz, Analyzing stability in wide-area network performance, Proceedings of the 1997 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.2-12, June 15-18, 1997, Seattle, Washington, United States
• 5: Paul Barford , Mark Crovella, Generating representative Web workloads for network and server performance evaluation, Proceedings of the 1998 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems, p.151-160, June 22-26, 1998, Madison, Wisconsin, United States
• 6: Braun, H. 1980. A simple method for testing goodness of fit in the presence of nuisance parameters. J. Royal Statis. Society. Series B (Methodological) 42, 1, 53--63.
• 7: Lee Breslau , Deborah Estrin , Kevin Fall , Sally Floyd , John Heidemann , Ahmed Helmy , Polly Huang , Steven McCanne , Kannan Varadhan , Ya Xu , Haobo Yu, Advances in Network Simulation, Computer, v.33 n.5, p.59-67, May 2000 [doi>10.1109/2.841785]
• 8: CAIDA. 2002. Internet measurement infrastructure. http://www.caida.org/analysis/performance/ measinfra/.
• 9: Jin Cao , William S. Cleveland , Dong Lin , Don X. Sun, On the nonstationarity of Internet traffic, Proceedings of the 2001 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.102-112, June 2001, Cambridge, Massachusetts, United States
• 10: Cao, J., Davis, D., Wiel, S., and Yu, B. 2000a. Time-varying network tomography: Router link data. J. Amer. Stat. Assoc. 95, 452 (Feb.), 1063--1075.
• 11: Cao, J., Wiel, S. V., Yu, B., and Zhu, Z. 2000b. A scalable method for estimating network traffic matrices. Bell Labs Tech. Rep..
• 12: Carter, R. and Crovella, M. 1996. Measuring bottleneck link speed in packet-switched networks. In In PERFORMANCE '96, the International Conference on Performance Theory, Measurement and Evaluation of Computer and Communication Systems.
• 13: Chandra, M., Singpurwalla, N. D., and Stephens, M. A. 1981. Kolmogorov statistics for tests of fit for the extreme-value and weibull distributions. J. Amer. Stat. Assoc. 76, 375 (Sept.), 729--731.
• 14: Cheshire, S. and Baker, M. 1995. Experiences with a wireless network in MosquitoNet. In Proceedings of the IEEE Hot Interconnects Symposium '95.
• 15: Mark E. Crovella , Azer Bestavros, Self-similarity in World Wide Web traffic: evidence and possible causes, Proceedings of the 1996 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.160-169, May 23-26, 1996, Philadelphia, Pennsylvania, United States
• 16: Allen B. Downey, Using pathchar to estimate Internet link characteristics, Proceedings of the 1999 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.222-223, May 01-04, 1999, Atlanta, Georgia, United States
• 17: Heyman, D., Tabatabi, A., Laksbman, T., and Heeke, H. 1994. Modeling teleconference traffic from vbr video coders. Proceedings ICC, IEEE, 1744--1748.
• 18: Anja Feldmann, BLT: Bi-layer tracing of HTTP and TCP&slash;IP, Computer Networks: The International Journal of Computer and Telecommunications Networking, v.33 n.1-6, p.321-335, June 2000
• 19: Anja Feldmann , Anna C. Gilbert , Polly Huang , Walter Willinger, Dynamics of IP traffic: a study of the role of variability and the impact of control, Proceedings of the conference on Applications, technologies, architectures, and protocols for computer communication, p.301-313, August 30-September 03, 1999, Cambridge, Massachusetts, United States
• 20: Sally Floyd , Vern Paxson, Difficulties in simulating the internet, IEEE/ACM Transactions on Networking (TON), v.9 n.4, p.392-403, August 2001 [doi>10.1109/90.944338]
• 21: Govindan, R., Alaettinoğlu, C., and Estrin, D. 1997. Self-configuring active network monitoring (SCAN). http://www.isi.edu/scon/Pubs/scan_proposal.ps.gz.
• 22: Gurenefelder, R., Cosmas, J. P., Manthrope, S., and Odinma-Okafor, A. 1991. Characterization of video codecs as autoregressive moving average processes and related queueing system performance. IEEE J. Selected Areas Comm. 9, 284--293.
• 23: Heffes, H. and Lucantoni, D. M. 1986. A markov modulated characterization of packetized voice and data traffic and related statistical multiplexer performance. IEEE J. Selected Areas Communications 4, 856--868.
• 24: Polly Huang , Anja Feldmann , Walter Willinger, A non-instrusive, wavelet-based approach to detecting network performance problems, Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement, November 01-02, 2001, San Francisco, California, USA [doi>10.1145/505202.505229]
• 25: Jacobson, V. 1997. Pathchar, April 1997 MSRI talk.
• 26: P. Kamath , K. Lan , J. Heidemann , J. Bannister , J. Touch, Generation of High Bandwidth Network Traffic Traces, Proceedings of the 10th IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems (MASCOTS'02), p.401, October 11-16, 2002
• 27: Lai, K. and Baker, M. 1999. Measuring bandwidth. In INFOCOM (1). 235--245.
• 28: Kevin Lai , Mary Baker, Measuring link bandwidths using a deterministic model of packet delay, Proceedings of the conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, p.283-294, August 28-September 01, 2000, Stockholm, Sweden
• 29: Lai, K. and Baker, M. 2001. Nettimer: A tool for measuring bottleneck link bandwidth. In Proceedings of the USENIX Symposium on Internet Technologies and Systems.
• 30: Lilliefors, H. W. 1969. On the kolmogorov-smirnov test for the exponential distribution with mean unknown. J. Amer. Stat. Assoc. 64, 325 (Mar.), 387--389.
• 31: David M. Lucantoni , Marcel F. Neuts , Amy R. Reibman, Methods for performance evaluation of VBR video traffic models, IEEE/ACM Transactions on Networking (TON), v.2 n.2, p.176-180, April 1994 [doi>10.1109/90.298435]
• 32: Maglaris, B., Anastassiou, D., P. Sen, G. K., and Robbins, J. D. 1988. Performance models of statistical multiplexing in packet video communications. IEEE Trans. Comm. 36, 7, 834--844.
• 33: Bruce A. Mah, An Empirical Model of HTTP Network Traffic, Proceedings of the INFOCOM '97. Sixteenth Annual Joint Conference of the IEEE Computer and Communications Societies. Driving the Information Revolution, p.592, April 09-11, 1997
• 34: Bruce A. Mah , Peter Sholander , Luis Martinez , Lawrence Tolendino, IPB: An Internet Protocol Benchmark Using Simulated Traffic, Proceedings of the 6th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, p.77, July 19-24, 1998
• 35: Mah, B. A. 1999. Pchar: Child of pathchar, presented at the DOE NGI testbed workshop, Berkeley, CA, 21 july 1999.
• 36: Massey and Jr., F. J. 1951. The Kolmogorov-Smirnov test of goodness of fit. J. Amer. Stat. Assoc. 46, 253 (Mar.), 68--78.
• 37: Mathis, M. and Mahdavi, J. 1996. Diagnosing Internet congestion with a transport layer performance tool. In Proceedings of INET '96. Montreal.
• 38: McCreary, S. and Claffy, K. 2000. Trends in wide area ip traffic patterns: A view from Ames Internet exchange. 13th ITC Specialist Seminar, 1--11.
• 39: Melamed, B. and Sengupta, B. 1992. Tes modeling of video traffic.
• 40: Minshall, G. 1997. Tcpdpriv, http://ita.ee.lbl.gov/html/contrib/tcpdpriv.html.
• 41: Nikolaidis, I. and Akyildiz, I. 1992. Source characterization and statistical multiplexing in atm networks. Tech. Rep. GIT-CC 92-24, Georgia Tech.
• 42: NLANR. 2001. Pma long traces archive, http://pma.nlanr.net/traces/long/.
• 43: Vern Paxson, Empirically derived analytic models of wide-area TCP connections, IEEE/ACM Transactions on Networking (TON), v.2 n.4, p.316-336, Aug. 1994 [doi>10.1109/90.330413]
• 44: Vern Edward Paxson, Measurements and analysis of end-to-end Internet dynamics, University of California at Berkeley, Berkeley, CA, 1998
• 45: Vern Paxson, End-to-end internet packet dynamics, IEEE/ACM Transactions on Networking (TON), v.7 n.3, p.277-292, June 1999 [doi>10.1109/90.779192]
• 46: Paxson, V. 2000. Internet Traffic Archive, http://www.acm.org/sigcomm/ita/.
• 47: Postel, J. and Reynolds, J. 1985. Rfc959.txt, file transfer protocol (FTP).
• 48: Sen, P., Maglaris, B., Rikli, N.-E., and Anastassiou, D. 1989. Models for packet switching of variable-bit-rate video sources. IEEE J. Selected Areas Comm. 7, 5, 865--869.
• 49: Smirnov, N. 1948. Table for estimating the goodness of fit of empirical distributions. Annals Math. Stat. 19, 2 (June), 279--281.
• 50: F. Donelson Smith , Félix Hernández Campos , Kevin Jeffay , David Ott, What TCP/IP protocol headers can tell us about the web, Proceedings of the 2001 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.245-256, June 2001, Cambridge, Massachusetts, United States
• 51: Vardi, Y. 1996. Network tomography: Estimating source-destination traffic intensities from link data. J. Amer. Stat. Assoc. 91, 433 (Mar.), 365--377.
• 52: Walter Willinger , Vern Paxson , Murad S. Taqqu, Self-similarity and heavy tails: structural modeling of network traffic, A practical guide to heavy tails: statistical techniques and applications, Birkhauser Boston Inc., Cambridge, MA, 1998
• 53: Yuksel, M., Sikdar, B., Vastola, K. S., and Szymanski, B. 2000. Workload generation for ns simulations of wide area net works and the Internet. In Proceedings of Communication Networks and Distributed Systems Modeling and Simulation Conference (CNDS) part of Western Multi-Conference (WMC). San Diego, CA, 93--98.
• 54: Yin Zhang , Nick Duffield, On the constancy of internet path properties, Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement, November 01-02, 2001, San Francisco, California, USA [doi>10.1145/505202.505228]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING
      I.6.0 General

Additional Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.0 General

General Terms:
Measurement

Keywords:
Model parameterization, Network, Traffic model