Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

While the problem of analyzing network traffic at the granularity of individual connections has seen considerable previous work and tool development, understanding traffic at a higher level - the structure of user-initiated sessions comprised of groups of related connections - remains much less explored. Some types of session structure, such as the coupling between an FTP control connection and the data connections it spawns, have prespecified forms, though the specifications do not guarantee how the forms appear in practice. Other types of sessions, such as a user reading email with a browser, only manifest empirically. Still other sessions might exist without us even knowing of their presence, such as a botnet zombie receiving instructions from its master and proceeding in turn to carry them out. We present algorithms rooted in the statistics of Poisson processes that can mine a large corpus of network connection logs to extract the apparent structure of application sessions embedded in the connections. Our methods are semi-automated in that we aim to present an analyst with high-quality information (expressed as regular expressions) reflecting different possible abstractions of an application's session structure. We develop and test our methods using traces from a large Internet site, finding diversity in the number of applications that manifest, their different session structures, and the presence of abnormal behavior. Our work has applications to traffic characterization and monitoring, source models for synthesizing network traffic, and anomaly detection.

REFERENCES

• 1: Paul Barford , Mark Crovella, Generating representative Web workloads for network and server performance evaluation, Proceedings of the 1998 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems, p.151-160, June 22-26, 1998, Madison, Wisconsin, United States [doi>10.1145/277851.277897]
• 2: A. Blum, D. Song, and S. Venkataraman. Detection of interactive stepping stones: Algorithms and confidence bounds. In 7th International Symposium on Recent Advances in Intrusion Detection (RAID 2004), pages 20--29, Sep 2004.
• 3: Jean-Chrysotome Bolot, End-to-end packet delay and loss behavior in the internet, Conference proceedings on Communications architectures, protocols and applications, p.289-298, September 13-17, 1993, San Francisco, California, United States [doi>10.1145/166237.166265]
• 4: K. Claffy, H.-W. Braun, and G. Polyzos. A parameterizable methodology for internet traffic flow profiling. IEEE JSAC, 13(8):1481--1494, 1995.
• 5: M. Costa, J. Crowcroft, M. Castro, A. Rowstron, C. Shannon, and J. Brown. Can we contain Internet worms? In Proc. HOTNETS, 2004.
• 6: Jedidiah R. Crandall , Frederic T. Chong, Minos: Control Data Attack Prevention Orthogonal to Memory Model, Proceedings of the 37th annual IEEE/ACM International Symposium on Microarchitecture, p.221-232, December 04-08, 2004, Portland, Oregon [doi>10.1109/MICRO.2004.26]
• 7: Mark E. Crovella , Azer Bestavros, Self-similarity in World Wide Web traffic: evidence and possible causes, Proceedings of the 1996 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.160-169, May 23-26, 1996, Philadelphia, Pennsylvania, United States [doi>10.1145/233013.233038]
• 8: W. Cui, V. Paxson, N. Weaver, and R. H. Katz. Protocol-Independent Adaptive Replay of Application Dialog. In Proc. NDSS, San Deigo, CA, Feb 2006.
• 9: P. Danzig, S. Jamin, R. Caceres, D. Mitzel, and D. Estrin. "an empirical workload model for driving wide-area tcpip network simulations". Internetworking: Research and Experience, 3(1):1--26, 1992.
• 10: Sally Floyd , Vern Paxson, Difficulties in simulating the internet, IEEE/ACM Transactions on Networking (TON), v.9 n.4, p.392-403, August 2001 [doi>10.1109/90.944338]
• 11: H. Fowler and W. Leland. Local area network traffic characteristics, with implications for broadband network congestion management. IEEE JSAC, 9(7):1139--1149, 1991.
• 12: Finite State Automata Utilities (version 6.2). http://odur.let.rug.nl/~ vannoord/Fsa/.
• 13: Graphviz - Graph Visualization Software. http://www.graphviz.org/.
• 14: S. Heimlich. Traffic characterization of the nsfnet national backbone. In Proc. Winter USENIX Conference, 1990.
• 15: R. Jain and S. Routhier. Packet trains - measurements and a new model for computer network traffic. IEEE JSAC, 4(6):986--995, 1986.
• 16: J. Kannan, J. Jung, V. Paxson, and C. E. Koksal. Detecting hidden causality in network connections. Technical report, University of California, Berkeley, 2005.
• 17: A. Kumar, V. Paxson, and N. Weaver. Exploiting underlying structure for detailed reconstruction of an internet-scale event. In Proc. ACM IMC, Oct 2005.
• 18: Will E. Leland , Murad S. Taqqu , Walter Willinger , Daniel V. Wilson, On the self-similar nature of Ethernet traffic (extended version), IEEE/ACM Transactions on Networking (TON), v.2 n.1, p.1-15, Feb. 1994 [doi>10.1109/90.282603]
• 19: Justin Ma , Kirill Levchenko , Christian Kreibich , Stefan Savage , Geoffrey M. Voelker, Unexpected means of protocol inference, Proceedings of the 6th ACM SIGCOMM on Internet measurement, October 25-27, 2006, Rio de Janeriro, Brazil [doi>10.1145/1177080.1177123]
• 20: Daniel A. Menascé , Virgilio A. F. Almeida , Rodrigo Fonseca , Marco A. Mendes, A methodology for workload characterization of E-commerce sites, Proceedings of the 1st ACM conference on Electronic commerce, p.119-128, November 03-05, 1999, Denver, Colorado, United States [doi>10.1145/336992.337024]
• 21: Vern Paxson, End-to-end Internet packet dynamics, Proceedings of the ACM SIGCOMM '97 conference on Applications, technologies, architectures, and protocols for computer communication, p.139-152, September 14-18, 1997, Cannes, France [doi>10.1145/263105.263155]
• 22: Andrew W. Moore , Denis Zuev, Internet traffic classification using bayesian analysis techniques, Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, June 06-10, 2005, Banff, Alberta, Canada [doi>10.1145/1064212.1064220]
• 23: J. Newsome and D. Song. Dynamic taint analysis for automatic detection, analysis, and signature generation of exploits on commodity software. In Proc. NDSS, 2005.
• 24: Carl Nuzman , Iraj Saniee , Wim Sweldens , Alan Weiss, A compound model for TCP connection arrivals for LAN and WAN applications, Computer Networks: The International Journal of Computer and Telecommunications Networking, v.40 n.3, p.319-337, 22 October 2002 [doi>10.1016/S1389-1286(02)00298-0]
• 25: Jitendra Padhye , Victor Firoiu , Don Towsley , Jim Kurose, Modeling TCP throughput: a simple model and its empirical validation, Proceedings of the ACM SIGCOMM '98 conference on Applications, technologies, architectures, and protocols for computer communication, p.303-314, August 31-September 04, 1998, Vancouver, British Columbia, Canada [doi>10.1145/285237.285291]
• 26: Vern Paxson, Empirically derived analytic models of wide-area TCP connections, IEEE/ACM Transactions on Networking (TON), v.2 n.4, p.316-336, Aug. 1994 [doi>10.1109/90.330413]
• 27: Vern Paxson, Automated packet trace analysis of TCP implementations, Proceedings of the ACM SIGCOMM '97 conference on Applications, technologies, architectures, and protocols for computer communication, p.167-179, September 14-18, 1997, Cannes, France [doi>10.1145/263105.263160]
• 28: Vern Paxson, End-to-end Internet packet dynamics, Proceedings of the ACM SIGCOMM '97 conference on Applications, technologies, architectures, and protocols for computer communication, p.139-152, September 14-18, 1997, Cannes, France [doi>10.1145/263105.263155]
• 29: Vern Paxson , Sally Floyd, Wide area traffic: the failure of Poisson modeling, IEEE/ACM Transactions on Networking (TON), v.3 n.3, p.226-244, June 1995 [doi>10.1109/90.392383]
• 30: TCP Ports List, UDP Ports List. http://seifried.org/security/ports.
• 31: The Protocol Informatics Project. http:www.//baselineresearch.net/PI/.
• 32: S. M. Ross. Introduction to Probability Models, 8th Edition. Academic Press, 2003.
• 33: F. Donelson Smith , Félix Hernández Campos , Kevin Jeffay , David Ott, What TCP/IP protocol headers can tell us about the web, Proceedings of the 2001 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.245-256, June 2001, Cambridge, Massachusetts, United States [doi>10.1145/378420.378789]
• 34: S. Staniford, S. Cheung, R. Crawford, M. Dilger, J. Frank, J. Hoagland, K. Levitt, C. Wee, R. Yip, and D. Zerkle. GrIDS - A graph-based intrusion detection system for large networks. In Proc. National Information Systems Security Conference, 1996.
• 35: S. Staniford-Chen , L. T. Heberlein, Holding intruders accountable on the Internet, Proceedings of the 1995 IEEE Symposium on Security and Privacy, p.39, May 08-10, 1995
• 36: Timbuktu Pro Remote Control Software. http://www.netopia.com/software/productst/b2/.
• 37: Walter Willinger , Murad S. Taqqu , Robert Sherman , Daniel V. Wilson, Self-similarity through high-variability: statistical analysis of ethernet LAN traffic at the source level, Proceedings of the conference on Applications, technologies, architectures, and protocols for computer communication, p.100-113, August 28-September 01, 1995, Cambridge, Massachusetts, United States [doi>10.1145/217382.217418]
• 38: Yinglian Xie , Vyas Sekar , David A. Maltz , Michael K. Reiter , Hui Zhang, Worm Origin Identification Using Random Moonwalks, Proceedings of the 2005 IEEE Symposium on Security and Privacy, p.242-256, May 08-11, 2005 [doi>10.1109/SP.2005.23]
• 39: Kunikazu Yoda , Hiroaki Etoh, Finding a Connection Chain for Tracing Intruders, Proceedings of the 6th European Symposium on Research in Computer Security, p.191-205, October 04-06, 2000
• 40: Yin Zhang , Lee Breslau , Vern Paxson , Scott Shenker, On the characteristics and origins of internet flow rates, Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications, August 19-23, 2002, Pittsburgh, Pennsylvania, USA [doi>10.1145/633025.633055]
• 41: Yin Zhang , Nick Duffield, On the constancy of internet path properties, Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement, November 01-02, 2001, San Francisco, California, USA [doi>10.1145/505202.505228]
• 42: Y. Zhang and V. Paxson. Detecting Stepping Stones. In Proc. 9th USENIX Security Symposium, Denver, CO, Aug 2000.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.2 Network Protocols

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.3 Network Operations
          Subjects: Network management

General Terms:
Algorithms, Measurement

Keywords:
anomaly detection, application sessions, traffic analysis