Specification-based anomaly detection: a new approach for detecting network intrusions

ABSTRACT

Unlike signature or misuse based intrusion detection techniques, anomaly detection is capable of detecting novel attacks. However, the use of anomaly detection in practice is hampered by a high rate of false alarms. Specification-based techniques have been shown to produce a low rate of false alarms, but are not as effective as anomaly detection in detecting novel attacks, especially when it comes to network probing and denial-of-service attacks. This paper presents a new approach that combines specification-based and anomaly-based intrusion detection, mitigating the weaknesses of the two approaches while magnifying their strengths. Our approach begins with state-machine specifications of network protocols, and augments these state machines with information about statistics that need to be maintained to detect anomalies. We present a specification language in which all of this information can be captured in a succinct manner. We demonstrate the effectiveness of the approach on the 1999 Lincoln Labs intrusion detection evaluation data, where we are able to detect all of the probing and denial-of-service attacks with a low rate of false alarms (less than 10 per day). Whereas feature selection was a crucial step that required a great deal of expertise and insight in the case of previous anomaly detection approaches, we show that the use of protocol specifications in our approach simplifies this problem. Moreover, the machine learning component of our approach is robust enough to operate without human supervision, and fast enough that no sampling techniques need to be employed. As further evidence of effectiveness, we present results of applying our approach to detect stealthy email viruses in an intranet environment.

REFERENCES

• 1: D. Anderson, T. Lunt, H. Javitz, A. Tamaru, and A. Valdes, Next-generation Intrusion Detection Expert System (NIDES): A Summary, SRI-CSL-95-07, SRI International, 1995.
• 2: T. Bowen, D. Chee, M. Segal, R. Sekar, P. Uppuluri, and T. Shanbhag, Building Survivable Systems: An Integrated Approach Based on Intrusion Detection and Confinement, DISCEX 2000.
• 3: P.K. Chan and S. Stolfo, Toward parallel and distributed learning by metalearning, AAAI workshop in Knowledge Discovery in Databases, 1993.
• 4: Dorothy E. Denning, An intrusion-detection model, IEEE Transactions on Software Engineering, v.13 n.2, p.222-232, Feb. 1987 [doi>10.1109/TSE.1987.232894]
• 5: Stephanie Forrest , Steven A. Hofmeyr , Anil Somayaji, Computer immunology, Communications of the ACM, v.40 n.10, p.88-96, Oct. 1997 [doi>10.1145/262793.262811]
• 6: Anup K. Ghosh , Aaron Schwartzbard , Michael Schatz, Learning Program Behavior Profiles for Intrusion Detection, Proceedings of the Workshop on Intrusion Detection and Network Monitoring, p.51-62, April 09-12, 1999
• 7: J. Haines, R. Lippmann, D. Fried, E. Tran, S. Boswell and M. Zissman, 1999 DARPA Intrusion Detection System Evaluation: Design and Procedures, MIT Lincoln Laboratory Technical Report TR-1062, 2001.
• 8: L. Heberlein et al, A Network Security Monitor, Symposium on Research Security and Privacy, 1990.
• 9: Judith Hochberg , Kathleen Jackson , Cathy Stallings , J. F. McClary , David DuBois , Josephine Ford, NADIR: an automated system for detecting network intrusion and misuse, Computers and Security, v.12 n.3, p.235-248, May 1993 [doi>10.1016/0167-4048(93)90110-Q]
• 10: G. Jakobson and M. Weissman, Alarm Correlation, IEEE Network, Vol. 7, No. 6., 1993.
• 11: C. Ko , M. Ruschitzka , K. Levitt, Execution monitoring of security-critical programs in distributed systems: a Specification-based approach, Proceedings of the 1997 IEEE Symposium on Security and Privacy, p.175, May 04-07, 1997
• 12: S. Kumar and E. Spafford, A Pattern-Matching Model for Intrusion Detection, Nat'l Computer Security Conference, 1994.
• 13: W. Lee and S. Stolfo, Data Mining Approaches for Intrusion Detection, USENIX Security Symposium, 1998.
• 14: R. Lippmann, D. Fried, I. Graf, J. Haines, K. Kendall, D. McClung, D. Weber, S. Webster, D. Wyschogrod, R. Cunningham and M. Zissman, Evaluating Intrusion Detection Systems: The 1998 DARPA Off-line Intrusion Detection Evaluation, Proceedings of the DARPA Information Survivability Conference and Exposition, 2000.
• 15: S. McCanne and V. Jacobson, The BSD Packet Filter: A New Architecture for User-level Packet Capture, Lawrence Berkeley Laboratory, Berkeley, CA, 1992.
• 16: B. Mukherjee, L. Heberlein and K. Levitt, Network Intrusion Detection, IEEE Network, May/June 1994.
• 17: V. Paxson, Bro: A System for Detecting Network Intruders in Real-Time, USENIX Security Symposium, 1998.
• 18: P. Porras and A. Valdes, Live Traffic Analysis of TCP/IP Gateways, Networks and Distributed Systems Security Symposium, 1998.
• 19: P. Porras and P. Neumann, EMERALD: Event Monitoring Enabled Responses to Anomalous Live Disturbances, National Information Systems Security Conference, 1997.
• 20: P. Porras and R. Kemmerer, Penetration State Transition Analysis:A Rule based Intrusion Detection Approach, Eighth Annual Computer Security Applications Conference, 1992.
• 21: Martin Roesch, Snort - Lightweight Intrusion Detection for Networks, Proceedings of the 13th USENIX conference on System administration, November 07-12, 1999, Seattle, Washington
• 22: R. Sekar , Y. Guang , S. Verma , T. Shanbhag, A high-performance network intrusion detection system, Proceedings of the 6th ACM conference on Computer and communications security, p.8-17, November 01-04, 1999, Kent Ridge Digital Labs, Singapore [doi>10.1145/319709.319712]
• 23: R. Sekar and P. Uppuluri, Synthesizing Fast Intrusion Prevention/Detection Systems from High-Level Specifications, USENIX Security Symposium, 1999.
• 24: Carol Taylor , Jim Alves-Foss, NATE: Network Analysis of Anomalous Traffic Events, a low-cost approach, Proceedings of the 2001 workshop on New security paradigms, September 10-13, 2001, Cloudcroft, New Mexico [doi>10.1145/508171.508186]
• 25: G. Vigna , R. A. Kemmerer, NetSTAT: A Network-Based Intrusion Detection Approach, Proceedings of the 14th Annual Computer Security Applications Conference, p.25, December 07-11, 1998

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.5 Security and Protection (D.4.6, K.4.2)
          Subjects: Unauthorized access (e.g., hacking, phreaking)

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.3 Network Operations
          Subjects: Network monitoring

General Terms:
Management, Security

Keywords:
anomaly detection, intrusion detection, network monitoring