A signal analysis of network traffic anomalies

ABSTRACT

Identifying anomalies rapidly and accurately is critical to the efficient operation of large computer networks. Accurately characterizing important classes of anomalies greatly facilitates their identification; however, the subtleties and complexities of anomalous traffic can easily confound this process. In this paper we report results of signal analysis of four classes of network traffic anomalies: outages, flash crowds, attacks and measurement failures. Data for this study consists of IP flow and SNMP measurements collected over a six month period at the border router of a large university. Our results show that wavelet filters are quite effective at exposing the details of both ambient and anomalous traffic. Specifically, we show that a pseudo-spline filter tuned at specific aggregation levels will expose distinct characteristics of each class of anomaly. We show that an effective way of exposing anomalies is via the detection of a sharp increase in the local variance of the filtered data. We evaluate traffic anomaly signals at different points within a network based on topological distance from the anomaly source or destination. We show that anomalies can be exposed effectively even when aggregated with a large amount of additional traffic. We also compare the difference between the same traffic anomaly signals as seen in SNMP and IP flow data, and show that the more coarse-grained SNMP data can also be used to expose anomalies effectively.

REFERENCES

• 1: Paul Barford , David Plonka, Characteristics of network traffic flow anomalies, Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement, November 01-02, 2001, San Francisco, California, USA [doi>10.1145/505202.505211]
• 2: Ramon Caceres, Measurements of Wide Area Internet TraffiC, University of California at Berkeley, Berkeley, CA, 1989
• 3: Vern Paxson, Fast, approximate synthesis of fractional Gaussian noise for generating self-similar network traffic, ACM SIGCOMM Computer Communication Review, v.27 n.5, p.5-18, Oct. 1997 [doi>10.1145/269790.269792]
• 4: Vern Edward Paxson, Measurements and analysis of end-to-end Internet dynamics, University of California at Berkeley, Berkeley, CA, 1998
• 5: Walter Willinger , Murad S. Taqqu , Robert Sherman , Daniel V. Wilson, Self-similarity through high-variability: statistical analysis of Ethernet LAN traffic at the source level, IEEE/ACM Transactions on Networking (TON), v.5 n.1, p.71-86, Feb. 1997 [doi>10.1109/90.554723]
• 6: P. Abry and D. Veitch, 'Wavelet analysis of long range dependent traffic," IEEE Transactions on Information Theory, vol. 44, no. 1, 1998.
• 7: Kimberly Claire Claffy, Internet traffic characterization, University of California at San Diego, La Jolla, CA, 1994
• 8: Cynthia S. Hood , Chuanyi Ji, Proactive Network Fault Detection, Proceedings of the INFOCOM '97. Sixteenth Annual Joint Conference of the IEEE Computer and Communications Societies. Driving the Information Revolution, p.1147, April 09-11, 1997
• 9: Irene Katzela , Mischa Schwartz, Schemes for fault identification in communication networks, IEEE/ACM Transactions on Networking (TON), v.3 n.6, p.753-764, Dec. 1995 [doi>10.1109/90.477721]
• 10: Amy Ward , Peter Glynn , Kathy Richardson, Internet service performance failure detection, ACM SIGMETRICS Performance Evaluation Review, v.26 n.3, p.38-43, Dec. 1998 [doi>10.1145/306225.306237]
• 11: Frank Feather , Dan Siewiorek , Roy Maxion, Fault detection in an Ethernet network using anomaly signature matching, Conference proceedings on Communications architectures, protocols and applications, p.279-288, September 13-17, 1993, San Francisco, California, United States
• 12: Jake D. Brutlag, Aberrant Behavior Detection in Time Series for Network Monitoring, Proceedings of the 14th USENIX conference on System administration, December 03-08, 2000, New Orleans, Louisiana
• 13: J. Toelle and O. Niggemann, 'Supporting intrusion detection by graph clustering and graph drawing," in Proceedings of Third International Workshop on Recent Advances in Intrusion Detection RAID 2000, Toulouse, France, October 2000.
• 14: K. Fox, R. Henning, J. Reed, and R. Simonian, 'A neural network approach towards intrusion detection," Tech. Rep., Harris Corporation, July 1990.
• 15: N. Ye, 'A markov chain model of temporal behavior for anomaly detection," in Workshop on Information Assurance and Security, West Point, NY, June 2000.
• 16: D. Moore, G. Voelker, and S. Savage, 'Inferring internet denial-of-service activity," in Proceedings of 2001 USENIX Security Symposium, Washington, DC, August 2001.
• 17: Vern Paxson, Bro: a system for detecting network intruders in real-time, Computer Networks: The International Journal of Computer and Telecommunications Networking, v.31 n.23-24, p.2435-2463, Dec. 1999 [doi>10.1016/S1389-1286(99)00112-7]
• 18: Jaeyeon Jung , Balachander Krishnamurthy , Michael Rabinovich, Flash crowds and denial of service attacks: characterization and implications for CDNs and web sites, Proceedings of the 11th international conference on World Wide Web, May 07-11, 2002, Honolulu, Hawaii, USA [doi>10.1145/511446.511485]
• 19: R. Manajan, S. Bellovin, S. Floyd, V. Paxson, S. Shenker, and J. Ioannidis, 'Controlling high bandwidth aggregates in the network," ACIRI Draft paper, February 2001.
• 20: Tobias Oetiker, MRTG: The Multi Router Traffic Grapher, Proceedings of the 12th Conference on Systems Administration, p.141-148, December 06-11, 1998
• 21: K. McCloghrie and F. Kastenholz, 'The interfaces group MIB,"IETF RFC 2863, June 2000.
• 22: Steve Romig, The OSU Flow-tools Package and CISCO NetFlow Logs, Proceedings of the 14th USENIX conference on System administration, December 03-08, 2000, New Orleans, Louisiana
• 23: Dave Plonka, FlowScan: A Network Traffic Flow Reporting and Visualization Tool, Proceedings of the 14th USENIX conference on System administration, December 03-08, 2000, New Orleans, Louisiana
• 24: Cisco IOS NetFlow, 'http://www.cisco.com/go/netfbw:' 2002.
• 25: I. Daubechies, B. Han, A. Ron, and Z. Shen, 'Framelets: MRA-based constructions of wavelet frames," Preprint: ftp://ftp:cs.wisc.edu/Approx/dhrs.ps, 2001.
• 26: Emmanuel Bacry, 'Lastwave:' http://www.cmap.polvtechnique.fr/~bacrv/ Last Wave.
• 27: P. Brockwell and R. Davis, Introduction to Time Series and Forecasting, Springer, 1996.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.3 Network Operations
          Subjects: Network monitoring

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.2 Network Protocols
          Subjects: Routing protocols
      C.2.5 Local and Wide-Area Networks
          Subjects: Internet (e.g., TCP/IP)
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Reliability, availability, and serviceability

General Terms:
Management, Measurement, Performance