Detection and classification of intrusions and faults using sequences of system calls

ABSTRACT

This paper investigates the use of sequences of system calls for classifying intrusions and faults induced by privileged processes in Unix. Classification is an essential capability for responding to an anomaly (attack or fault), since it gives the ability to associate appropriate responses to each anomaly type. Previous work using the well known dataset from the University of New Mexico (UNM) has demonstrated the usefulness of monitoring sequences of system calls for detecting anomalies induced by processes corresponding to several Unix Programs, such as sendmail, lpr, ftp, etc. Specifically, previous work has shown that the Anomaly Count of a running process, i.e., the number of sequences spawned by the process which are not found in the corresponding dictionary of normal activity for the Program, is a valuable feature for anomaly detection. To achieve Classification, in this paper we introduce the concept of Anomaly Dictionaries, which are the sets of anomalous sequences for each type of anomaly. It is verified that Anomaly Dictionaries for the UNM's sendmail Program have very little overlap, and can be effectively used for Anomaly Classification. The sequences in the Anomalous Dictionary enable a description of Self for the Anomalies, analogous to the definition of Self for Privileged Programs given by the Normal Dictionaries. The dependence of Classification Accuracy with sequence length is also discussed. As a side result, it is also shown that a hybrid scheme, combining the proposed classification strategy with the original Anomaly Counts can lead to a substantial improvement in the overall detection rates for the sendmail dataset. The methodology proposed is rather general, and can be applied to any situation where sequences of symbols provide an effective characterization of a phenomenon.

REFERENCES

• 1: João B. D. Cabrera , B. Ravichandran , Raman K. Mehra, Statistical Traffic Modeling for Network Intrusion Detection, Proceedings of the 8th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, p.466, August 29-September 01, 2000
• 2: W. Cohen. Fast effective rule induction. In Proceedings of the Twelfth International Conference on Machine Learning, 1995.
• 3: E. Eskin, W. Lee, and S. Stolfo. Modeling system call for intrusion detection using dynamic window sizes. In Proceedings of the 2001 DARPA Information Survivability Conference & Exposition, Anaheim, CA, June 2001.
• 4: Stephanie Forrest , Steven A. Hofmeyr , Anil Somayaji , Thomas A. Longstaff, A Sense of Self for Unix Processes, Proceedings of the 1996 IEEE Symposium on Security and Privacy, p.120, May 06-08, 1996
• 5: Keinosuke Fukunaga, Introduction to statistical pattern recognition (2nd ed.), Academic Press Professional, Inc., San Diego, CA, 1990
• 6: Anup K. Ghosh , Aaron Schwartzbard , Michael Schatz, Learning Program Behavior Profiles for Intrusion Detection, Proceedings of the Workshop on Intrusion Detection and Network Monitoring, p.51-62, April 09-12, 1999
• 7: G. G. Helmer, J. S. K. Wong, V. Hanavar, and L. Miller. Intelligent agents for intrusion detection. In Proceedings of the IEEE Information Technology Conference, Syracuse, NY, 1998.
• 8: S. A. Hofmeyr, S. Forrest, and A. Somayaji. Intrusion detection using sequences of system calls. Journal of Computer Security, 6:151-180, 1998.
• 9: Sushil Jajodia , Catherine D. McCollum , Paul Ammann, Trusted recovery, Communications of the ACM, v.42 n.7, p.71-75, July 1999 [doi>10.1145/306549.306580]
• 10: C. Ko, G. Fink, and K. Levitt. Automated detection of vulnerabilities in privileged programs by execution monitoring. In Proceedings of the 10th Annual Computer Security Applications Conference, pages 134-144, December 1994.
• 11: Wenke Lee , Salvatore J. Stolfo , Kui W. Mok, Adaptive Intrusion Detection: A Data Mining Approach, Artificial Intelligence Review, v.14 n.6, p.533-567, December 1, 2000 [doi>10.1023/A:1006624031083]
• 12: W. Lee, S. J. Stolfo, and P. K. Chan. Learning Patterns from Unix Process Execution Traces for Intrusion Detection. In Proceedings of the AAAI Workshop on AI Methods in Fraud and Risk Management, pages 50-56, July 1997.
• 13: Wenke Lee , Dong Xiang, Information-Theoretic Measures for Anomaly Detection, Proceedings of the 2001 IEEE Symposium on Security and Privacy, p.130, May 14-16, 2001
• 14: Carla Marceau, Characterizing the behavior of a program using multiple-length N-grams, Proceedings of the 2000 workshop on New security paradigms, p.101-110, September 18-21, 2000, Ballycotton, County Cork, Ireland [doi>10.1145/366173.366197]
• 15: Ravi Mukkamala , Jason Gagnon , Sushil Jajodia, Integrating Data Mining Techniques with Intrusion Detection Methods, Proceedings of the IFIP WG 11.3 Thirteenth International Conference on Database Security: Research Advances in Database and Information Systems Security, p.33-46, July 26-28, 1999
• 16: Peter G. Neumann , Phillip A. Porras, Experience with EMERALD to Date, Proceedings of the Workshop on Intrusion Detection and Network Monitoring, p.73-80, April 09-12, 1999
• 17: Matthew Stillerman , Carla Marceau , Maureen Stillman, Intrusion detection for distributed applications, Communications of the ACM, v.42 n.7, p.62-69, July 1999 [doi>10.1145/306549.306577]
• 18: S. Warrender, S. Forrest, and B. Pearlmutter. Detecting intrusions using system calls: Alternative data models. In Proceedings of the IEEE Symposium on Security and Privacy, pages 133-145, 1999.
• 19: A. Wespi, H. Debar, M. Dacier, and M. Nassehi. Fixed-vs.variable-length patterns for detecting suspicious process behavior. Journal of Computer Security, 8:159-181, 2000.

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.5 Security and Protection (D.4.6, K.4.2)

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining

General Terms:
Performance, Security