A fast string-matching algorithm for network processor-based intrusion detection system

ABSTRACT

Network intrusion detection systems (NIDSs) are one of the latest developments in security. The matching of packet strings against collected signatures dominates signature-based NIDS performance. Network processors are also one of the fastest growing segments of the semiconductor market, because they are designed to provide scalable and flexible solutions that can accommodate change quickly and economically. This work presents a fast string-matching algorithm (called FNP) over the network processor platform that conducts matching sets of patterns in parallel. This design also supports numerous practical features such as case-sensitive string matching, signature prioritization, and multiple-content signatures. This efficient multiple-pattern matching algorithm utilizes the hardware facilities provided by typical network processors instead of employing the external lookup co-processors. To verify the efficiency and practicability of the proposed algorithm, it was implemented on the Vitesse IQ2000 network processor platform. The searching patterns used in the present experiments are derived from the well-known Snort ruleset cited by most open-source and commercial NIDSs. This work shows that combining our string-matching methodology, hashing engine supported by most network processors, and characteristics of current Snort signatures frequently improves performance and reduces number of memory accesses compared to conventional string-matching algorithms. Another contribution of this work is to highlight that, besides total number of searching patterns, shortest pattern length is also a major influence on NIDS multipattern matching algorithm performance.

REFERENCES

• 1: Alfred V. Aho , Margaret J. Corasick, Efficient string matching: an aid to bibliographic search, Communications of the ACM, v.18 n.6, p.333-340, June 1975 [doi>10.1145/360825.360855]
• 2: Anagnostakis, K. G., Markatos, E. P., Antonatos, S., and Polychronakis, M. 2003. E2xB: A domain-specific string matching algorithm for intrusion detection. Proceedings of the 18th IFIP International Information Security Conference (SEC2003).
• 3: Robert S. Boyer , J. Strother Moore, A fast string searching algorithm, Communications of the ACM, v.20 n.10, p.762-772, Oct. 1977 [doi>10.1145/359842.359859]
• 4: Coit, C. J., Staniford, S., and McAlerney, J. 2001. Towards faster patern matching for intrusion detection or exceeding the speed of snort. Proceedings of the 2nd DARPA Information Survivability Conference and Exposition (DISCEX II).
• 5: Beate Commentz-Walter, A String Matching Algorithm Fast on the Average, Proceedings of the 6th Colloquium, on Automata, Languages and Programming, p.118-132, July 16-20, 1979
• 6: DEFCON. http://www.shmoo.com/cctf/
• 7: Mike Fisk , George Varghese, Fast Content-Based Packet Handling for Intrusion Detection, University of California at San Diego, La Jolla, CA, 2001
• 8: Horspool, R. N. 1980. Practical fast searching in strings. Softw. Pract. Experience, 10, 6, 501--506.
• 9: Intel Network Processor. http://www.intel.com/design/network/products/npfamily/
• 10: Kim, S. and Kim, Y. 1999. A fast multiple string-pattern matching algorithm. Proceedings of the 17th AoM/IAoM Inernational Conference on Computer Science.
• 11: Markatos, E. P., Antonatos, S., Polychronakis, M., and Anagnostakis, K. G. 2002. ExB: Exclusion-based signature matching for intrusion detection. Proceedings of the IASTED International Conference on Communications and Computer Networks (CCN), Cambridge, USA. 146--152.
• 12: Norton, M. and Roelker, D. 2002. Snort 2.0: Detection Revised. http://www.sourcefire.com/technology/whitepapers/sf_snort20_ detection_rvstd.pdf
• 13: P. Paulin , F. Karim , P. Bromley, Network processors: a perspective on market requirements, processor architectures and embedded S/W tools, Proceedings of the conference on Design, automation and test in Europe, p.420-429, March 2001, Munich, Germany
• 14: Martin Roesch, Snort - Lightweight Intrusion Detection for Networks, Proceedings of the 13th USENIX conference on System administration, November 07-12, 1999, Seattle, Washington
• 15: Shah, N. and Keutzer, K. 2002. Network processors: Origin of species, in Proceedings of ISCIS XVII, the Seventeenth International Symposium on Computer and Information Sciences.
• 16: Shah N., Plishker, W., Keutzer, K. 2003. NP-Click: A programming model for the intel IXP1200. In 2nd Workshop on Network Processors (NP-2) at the 9th International Symposium on High Performance Computer Architecture (HPCA-9), Anaheim, CA.
• 17: Snort. http://www.snort.org/
• 18: Spirent Communications, http://smartbits.spirentcom.com/
• 19: Vitesse. http://www.vitesse.com/
• 20: Watson, B. W. 1994. The performance of single-keyword and multiple-keyword pattern matching algorithms. Tech. rep. 94/19, Eindhoven University of Technology.
• 21: Whitehats. http://www.whitehats.com/
• 22: Wu, S. and Manber, U. 1994. A fast algorithm for multi-pattern searching. Tech. rep. TR94-17, Department of Computer Science, University of Arizona.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.0 General
          Subjects: Security and protection (e.g., firewalls)

Additional Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.4 Applications

General Terms:
Algorithms, Design, Performance

Keywords:
Intrusion detection, network, pattern matching, processor