Enhancing byte-level network intrusion detection signatures with context

ABSTRACT

Many network intrusion detection systems (NIDS) use byte sequences as signatures to detect malicious activity. While being highly efficient, they tend to suffer from a high false-positive rate. We develop the concept of contextual signatures as an improvement of string-based signature-matching. Rather than matching fixed strings in isolation, we augment the matching process with additional context. When designing an efficient signature engine for the NIDS bro, we provide low-level context by using regular expressions for matching, and high-level context by taking advantage of the semantic information made available by bro's protocol analysis and scripting language. Therewith, we greatly enhance the signature's expressiveness and hence the ability to reduce false positives. We present several examples such as matching requests with replies, using knowledge of the environment, defining dependencies between signatures to model step-wise attacks, and recognizing exploit scans.To leverage existing efforts, we convert the comprehensive signature set of the popular freeware NIDS snort into bro's language. While this does not provide us with improved signatures by itself, we reap an established base to build upon. Consequently, we evaluate our work by comparing to snort, discussing in the process several general problems of comparing different NIDSs.

REFERENCES

• 1: arachNIDS. http://whitehats.com/ids/.
• 2: Web archive of versions of software and signatures used in this paper. http://www.net.in.tum.de/~robin/ccs03.
• 3: Stefan Axelsson, The base-rate fallacy and the difficulty of intrusion detection, ACM Transactions on Information and System Security (TISSEC), v.3 n.3, p.186-205, Aug. 2000 [doi>10.1145/357830.357849]
• 4: Rebecca Gurley Bace, Intrusion detection, Macmillan Publishing Co., Inc., Indianapolis, IN, 1999
• 5: Bro: A System for Detecting Network Intruders in Real-Time. http://www.icir.org/vern/bro-info.html.
• 6: Bugtraq. http://www.securityfocus.com/bid/1187.
• 7: CERT Advisory CA-2002-27 Apache/mod_ssl Worm. http://www.cert.org/advisories/CA-2002-27.html.
• 8: C. J. Coit, S. Staniford, and J. McAlerney. Towards Faster Pattern Matching for Intrusion Detection or Exceeding the Speed of Snort. In Proc. 2nd DARPA Information Survivability Conference and Exposition, June 2001.
• 9: Common Vulnerabilities and Exposures. http://www.cve.mitre.org.
• 10: H. Debar and B. Morin. Evaluation of the Diagnostic Capabilities of Commercial Intrusion Detection Systems. In Proc. Recent Advances in Intrusion Detection, number 2516 in Lecture Notes in Computer Science. Springer-Verlag, 2002.
• 11: R. F. et. al. Hypertext transfer protocol -- http/1.1. Request for Comments 2616, June 1999.
• 12: Mike Fisk , George Varghese, Fast Content-Based Packet Handling for Intrusion Detection, University of California at San Diego, La Jolla, CA, 2001
• 13: Fyodor. Remote OS detection via TCP/IP Stack Finger Printing. Phrack Magazine, 8(54), 1998.
• 14: J. Haines, L. Rossey, R. Lippmann, and R. Cunnigham. Extending the 1999 Evaluation. In Proc. 2nd DARPA Information Survivability Conference and Exposition, June 2001.
• 15: M. Hall and K. Wiley. Capacity Verification for High Speed Network Intrusion Detection Systems. In Proc. Recent Advances in Intrusion Detection, number 2516 in Lecture Notes in Computer Science. Springer-Verlag, 2002.
• 16: M. Handley, C. Kreibich, and V. Paxson. Network intrusion detection: Evasion, traffic normalization, and end-to-end protocol semantics. In Proc. 10th USENIX Security Symposium, Washington, D.C., August 2001.
• 17: J. Heering , P. Klint , J. Rekers, Incremental generation of lexical scanners, ACM Transactions on Programming Languages and Systems (TOPLAS), v.14 n.4, p.490-520, Oct. 1992 [doi>10.1145/133233.133240]
• 18: John E. Hopcroft , Jeffrey D. Ullman, Introduction To Automata Theory, Languages, And Computation, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1990
• 19: K. Jackson. Intrusion detection system product survey. Technical Report LA-UR-99-3883, Los Alamos National Laboratory, June 1999.
• 20: U. Lindqvist and P. A. Porras. Detecting computer and network misuse through the production-based expert system toolset (P-BEST). In Proc. IEEE Symposium on Security and Privacy. IEEE Computer Society Press, May 1999.
• 21: Richard Lippmann , Joshua W. Haines , David J. Fried , Jonathan Korba , Kumar Das, Analysis and Results of the 1999 DARPA Off-Line Intrusion Detection Evaluation, Proceedings of the Third International Workshop on Recent Advances in Intrusion Detection, p.162-182, October 02-04, 2000
• 22: Richard Lippmann , Joshua W. Haines , David J. Fried , Jonathan Korba , Kumar Das, The 1999 DARPA off-line intrusion detection evaluation, Computer Networks: The International Journal of Computer and Telecommunications Networking, v.34 n.4, p.579-595, Oct. 2000 [doi>10.1016/S1389-1286(00)00139-0]
• 23: R. Lippmann, S. Webster, and D. Stetson. The Effect of Identifying Vulnerabilities and Patching Software on the Utility of Network Intrusion Detection. In Proc. Recent Advances in Intrusion Detection, number 2516 in Lecture Notes in Computer Science. Springer-Verlag, 2002.
• 24: Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory, ACM Transactions on Information and System Security (TISSEC), v.3 n.4, p.262-294, Nov. 2000 [doi>10.1145/382912.382923]
• 25: Vern Paxson, Bro: a system for detecting network intruders in real-time, Computer Networks: The International Journal of Computer and Telecommunications Networking, v.31 n.23-24, p.2435-2463, Dec. 1999 [doi>10.1016/S1389-1286(99)00112-7]
• 26: P. A. Porras and P. G. Neumann. EMERALD: Event monitoring enabling responses to anomalous live disturbances. In National Information Systems Security Conference, Baltimore, MD, October 1997.
• 27: T. H. Ptacek and T. N. Newsham. Insertion, evasion, and denial of service: Eluding network intrusion detection. Technical report, Secure Networks, Inc., January 1998.
• 28: Marcus J. Ranum , Kent Landfield , Michael T. Stolarchuk , Mark Sienkiewicz , Andrew Lambeth , Eric Wall, Implementing a Generalized Tool for Network Monitoring, Proceedings of the 11th Conference on Systems Administration, p.1-8, October 26-31, 1997
• 29: Martin Roesch, Snort - Lightweight Intrusion Detection for Networks, Proceedings of the 13th USENIX conference on System administration, November 07-12, 1999, Seattle, Washington
• 30: R. Sekar and P. Uppuluri. Synthesizing fast intrusion prevention/detection systems from high-level specifications. In Proc. 8th USENIX Security Symposium. USENIX Association, August 1999.
• 31: Umesh Shankar , Vern Paxson, Active Mapping: Resisting NIDS Evasion without Altering Traffic, Proceedings of the 2003 IEEE Symposium on Security and Privacy, p.44, May 11-14, 2003
• 32: Steven T. Eckmann. Translating Snort rules to STATL scenarios. In Proc. Recent Advances in Intrusion Detection, October 2001.
• 33: tcpdump. http://www.tcpdump.org.
• 34: Valgrind. http://developer.kde.org/~sewardj.
• 35: G. Vigna, S. Eckmann, and R. Kemmerer. The STAT Tool Suite. In Proc. 1st DARPA Information Survivability Conference and Exposition, Hilton Head, South Carolina, January 2000. IEEE Computer Society Press.
• 36: Giovanni Vigna , Richard A. Kemmerer, NetSTAT: a network-based intrusion detection system, Journal of Computer Security, v.7 n.1, p.37-71, Sept. 1999
• 37: Whisker. http://www.wiretrip.net/rfp.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.0 General
          Subjects: Security and protection (e.g., firewalls)

General Terms:
Performance, Security

Keywords:
bro, evaluation, network intrusion detection, pattern matching, security, signatures, snort