Enriching network security analysis with time travel

ABSTRACT

In many situations it can be enormously helpful to archive the raw contents of a network traffic stream to disk, to enable later inspection of activity that becomes interesting only in retrospect. We present a Time Machine (TM) for network traffic that provides such a capability. The TM leverages the heavy-tailed nature of network flows to capture nearly all of the likely-interesting traffic while storing only a small fraction of the total volume. An initial proof-of-principle prototype established the forensic value of such an approach, contributing to the investigation of numerous attacks at a site with thousands of users. Based on these experiences, a rearchitected implementation of the system provides flexible, highperformance traffic stream capture, indexing and retrieval, including an interface between the TM and a real-time network intrusion detection system (NIDS). The NIDS controls the TM by dynamically adjusting recording parameters, instructing it to permanently store suspicious activity for offline forensics, and fetching traffic from the past for retrospective analysis. We present a detailed performance evaluation of both stand-alone and joint setups, and report on experiences with running the system live in high-volume environments.

REFERENCES

• 1: ANDERSON, E., AND ARLITT, M. Full Packet Capture and Offline Analysis on 1 and 10 Gb/s Networks. Tech. Rep. HPL-2006-156, HP Labs, 2006.
• 2: ANTONELLI, C., CO, K., M FIELDS, AND HONEYMAN, P. Cryptographic Wiretapping at 100 Megabits. In SPIE 16th Int. Symp. on Aerospace Defense Sensing, Simulation, and Controls. (2002).
• 3: Sirish Chandrasekaran , Michael Franklin, Remembrance of streams past: overload-sensitive management of archived streams, Proceedings of the Thirtieth international conference on Very large data bases, p.348-359, August 31-September 03, 2004, Toronto, Canada
• 4: ClearSight Networks. http://www.clearsightnet.com.
• 5: CNET NEWS. Another suspected NASA hacker indicted. http://www.news.com/2102-7350_3-6140001.html.
• 6: CoMo. http://como.sourceforge.net.
• 7: Evan Cooke , Andrew Myrick , David Rusek , Farnam Jahanian, Resource-aware multi-format network security data storage, Proceedings of the 2006 SIGCOMM workshop on Large-scale attack defense, p.177-184, September 11-15, 2006, Pisa, Italy [doi>10.1145/1162666.1162677]
• 8: Chuck Cranor , Theodore Johnson , Oliver Spataschek , Vladislav Shkapenyuk, Gigascope: a stream database for network applications, Proceedings of the 2003 ACM SIGMOD international conference on Management of data, June 09-12, 2003, San Diego, California [doi>10.1145/872757.872838]
• 9: Peter J. Desnoyers , Prashant Shenoy, Hyperion: high volume stream archival for retrospective querying, 2007 USENIX Annual Technical Conference on Proceedings of the USENIX Annual Technical Conference, p.1-14, June 17-22, 2007, Santa Clara, CA
• 10: Holger Dreger , Anja Feldmann , Vern Paxson , Robin Sommer, Operational experiences with high-volume network intrusion detection, Proceedings of the 11th ACM conference on Computer and communications security, October 25-29, 2004, Washington DC, USA [doi>10.1145/1030083.1030086]
• 11: George W. Dunlap , Samuel T. King , Sukru Cinar , Murtaza A. Basrai , Peter M. Chen, ReVirt: enabling intrusion analysis through virtual-machine logging and replay, Proceedings of the 5th symposium on Operating systems design and implementation

  Due to copyright restrictions we are not able to make the PDFs for this conference available for downloading

  , December 09-11, 2002, Boston, Massachusetts [doi>10.1145/1060289.1060309]
• 12: ENDACE MEASUREMENT SYSTEMS. http://www.endace.com/, 2008.
• 13: Jose M. Gonzalez , Vern Paxson , Nicholas Weaver, Shunting: a hardware/software architecture for flexible, high-performance network intrusion prevention, Proceedings of the 14th ACM conference on Computer and communications security, October 28-31, 2007, Alexandria, Virginia, USA [doi>10.1145/1315245.1315264]
• 14: Intelica Networks. http://www.intelicanetworks.com.
• 15: Stefan Kornexl , Vern Paxson , Holger Dreger , Anja Feldmann , Robin Sommer, Building a time machine for efficient recording and retrieval of high-volume network traffic, Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement, p.23-23, October 19-21, 2005, Berkeley, CA
• 16: Kevin P. Mc Grath , John Nelson, Monitoring & Forensic Analysis forWireless Networks, Proceedings of the International Conference on Internet Surveillance and Protection, p.4, August 26-28, 2006 [doi>10.1109/ICISP.2006.21]
• 17: Kihong Park , Gitae Kim , Mark Crovella, On the relationship between file sizes, transport protocols, and self-similar network traffic, Proceedings of the 1996 International Conference on Network Protocols (ICNP '96), p.171, October 29-November 01, 1996
• 18: Vern Paxson, Bro: a system for detecting network intruders in real-time, Computer Networks: The International Journal of Computer and Telecommunications Networking, v.31 n.23-24, p.2435-2463, Dec. 1999 [doi>10.1016/S1389-1286(99)00112-7]
• 19: Vern Paxson , Sally Floyd, Wide area traffic: the failure of Poisson modeling, IEEE/ACM Transactions on Networking (TON), v.3 n.3, p.226-244, June 1995 [doi>10.1109/90.392383]
• 20: Miroslav Ponec , Paul Giura , Hervé Brönnimann , Joel Wein, Highly efficient techniques for network forensics, Proceedings of the 14th ACM conference on Computer and communications security, October 28-31, 2007, Alexandria, Virginia, USA [doi>10.1145/1315245.1315265]
• 21: Frederick Reiss , Kurt Stockinger , Kesheng Wu , Arie Shoshani , Joseph M. Hellerstein, Enabling Real-Time Querying of Live and Historical Stream Data, Proceedings of the 19th International Conference on Scientific and Statistical Database Management, p.28, July 09-11, 2007 [doi>10.1109/SSDBM.2007.34]
• 22: Martin Roesch, Snort - Lightweight Intrusion Detection for Networks, Proceedings of the 13th USENIX conference on System administration, November 07-12, 1999, Seattle, Washington
• 23: SHANMUGASUNDARAM, K., MEMON, N., SAVANT, A., AND BRÖNNIMANN, H. ForNet: A Distributed Forensics Network. In Proc. Workshop on Math. Methods, Models and Architectures for Comp. Networks Security (2003).
• 24: SOMMER, R. Viable Network Intrusion Detection in High-Performance Environments. PhD thesis, TU München, 2005.
• 25: Robin Sommer , Vern Paxson, Exploiting Independent State For Network Intrusion Detection, Proceedings of the 21st Annual Computer Security Applications Conference, p.59-71, December 05-09, 2005 [doi>10.1109/CSAC.2005.24]
• 26: VALLENTIN, M., SOMMER, R., LEE, J., LERES, C., PAXSON, V., AND TIERNEY, B. The NIDS Cluster: Scalable, Stateful Network Intrusion Detection on Commodity Hardware. In Proc. 10th Int. Symp. Recent Advances in Intrusion Detection (RAID) (2007).
• 27: Jörg Wallerich , Holger Dreger , Anja Feldmann , Balachander Krishnamurthy , Walter Willinger, A methodology for studying persistency aspects of internet flows, ACM SIGCOMM Computer Communication Review, v.35 n.2, April 2005 [doi>10.1145/1064413.1064417]

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.3 Network Operations
          Subjects: Network monitoring

General Terms:
Measurement, Performance, Security

Keywords:
forensics, intrusion detection, packet capture