Evaluation of Intrusion Detection Systems Under a Resource Constraint

ABSTRACT

An intrusion detection system plays an important role in a firm's overall security protection. Its main purpose is to identify potentially intrusive events and alert the security personnel to the danger. A typical intrusion detection system, however, is known to be imperfect in detection of intrusive events, resulting in high false-alarm rates. Nevertheless, current intrusion detection models unreasonably assume that upon alerts raised by a system, an information security officer responds to all alarms without any delay and avoids damages of hostile activities. This assumption of responding to all alarms with no time lag is often impracticable. As a result, the benefit of an intrusion detection system can be overestimated by current intrusion detection models. In this article, we extend previous models by including an information security officer's alarm inspection under a constraint as a part of the process in determining the optimal intrusion detection policy. Given a potentially hostile environment for a firm, in which the intrusion rates and costs associated with intrusion and security officers' inspection can be estimated, we outline a framework to establish the optimal operating points for intrusion detection systems under security officers' inspection constraint. The optimal solution to the model will provide not only a basis of better evaluation of intrusion detection systems but also useful insights into operations of intrusion detection systems. The firm can estimate expected benefits for running intrusion detection systems and establish a basis for increase in security personnel to relax security officers' inspection constraint.

REFERENCES

• 1: R. Anderson, Why Information Security is Hard-An Economic Perspective, Proceedings of the 17th Annual Computer Security Applications Conference, p.358, December 10-14, 2001
• 2: Stefan Axelsson, The base-rate fallacy and the difficulty of intrusion detection, ACM Transactions on Information and System Security (TISSEC), v.3 n.3, p.186-205, Aug. 2000 [doi>10.1145/357830.357849]
• 3: Cardenas, A. A., Ramezani, V., and Baras, J. S. 2003. HMM sequential hypothesis tests for intrusion detection in MANETs. Technical Report 2003-47, Department of Electrical and Computer Engineering, Maryland University.
• 4: Huseyin Cavusoglu , Srinivasan Raghunathan, Configuration of Detection Software: A Comparison of Decision and Game Theory Approaches, Decision Analysis, v.1 n.3, p.131-148, September 2004 [doi>10.1287/deca.1040.0022]
• 5: Robert Durst , Terrence Champion , Brian Witten , Eric Miller , Luigi Spagnuolo, Testing and evaluating computer intrusion detection systems, Communications of the ACM, v.42 n.7, p.53-61, July 1999 [doi>10.1145/306549.306571]
• 6: Endorf, C., Schultz, E., and Mellander, J. 2004. Intrusion Detection & Prevention. Emeryville, CA: McGraw-Hill/Osborne.
• 7: John E. Gaffney Jr , Jacob W. Ulvila, Evaluation of Intrusion Detectors: A Decision Theory Approach, Proceedings of the 2001 IEEE Symposium on Security and Privacy, p.50, May 14-16, 2001
• 8: Kemmerer, R. A. and Vigna, G. 2002. Intruder detection: A brief history and overview. Secur. Privacy, Suppl. Comput. 0, 27--30.
• 9: Kent, S. 2000. On the trail of intrusions into information systems. IEEE Spectrum 37, 12, 52--56.
• 10: Wenke Lee , Wei Fan , Matthew Miller , Salvatore J. Stolfo , Erez Zadok, Toward cost-sensitive modeling for intrusion detection and response, Journal of Computer Security, v.10 n.1-2, p.5-22, 2002
• 11: Richard Lippmann , Joshua W. Haines , David J. Fried , Jonathan Korba , Kumar Das, Analysis and Results of the 1999 DARPA Off-Line Intrusion Detection Evaluation, Proceedings of the Third International Workshop on Recent Advances in Intrusion Detection, p.162-182, October 02-04, 2000
• 12: Lippmann, R. P., Fried, D. J., Graf, I., Haines, J. W., Kendall, K. R., McClung, D., Weber, D., Webster, S. E., Wyschogrod, D., Cunningham, R. K., and Zissman, M. A. 2000. Evaluating intrusion detection systems: The 1998 DARPA off-line intrusion detection evaluation. In Proceedings of DARPA Information Survivability Conference and Exposition (DISCEX'00). Vol. 2. Hilton Head, SC, 1012--1026.
• 13: Metz, C. E. 1978. Basic principles of ROC analysis. Seminars in Nuclear Medicine 8, 4, 283--298.
• 14: Peter G. Neumann , Phillip A. Porras, Experience with EMERALD to Date, Proceedings of the Workshop on Intrusion Detection and Network Monitoring, p.73-80, April 09-12, 1999
• 15: Donn B. Parker, Fighting Computer Crime, Macmillan Library Reference, 1983
• 16: Porras, P. A. and Neumann, P. G. 1997. EMERALD: Event monitoring enabling responses to anomalous live disturbances. In Proceedings of the 20th NIST-NCSC National Information Systems Security Conference (NISSC'97). Baltimore, MD, 353--365.
• 17: Paul E. Proctor, Practical Intrusion Detection Handbook, Prentice Hall PTR, Upper Saddle River, NJ, 2000
• 18: Ryu, Y. U. and Yue, W. T. 2003. A risk-based evaluation of intrusion detection systems in the presence of the base-rate fallacy. Working paper, Department of Information Systems and Operations Management, School of Management, The University of Texas at Dallas.
• 19: Salvatore J. Stolfo , Wenke Lee , Philip K. Chan , Wei Fan , Eleazar Eskin, Data mining-based intrusion detectors: an overview of the columbia IDS project, ACM SIGMOD Record, v.30 n.4, December 2001 [doi>10.1145/604264.604267]
• 20: The Snort Project. 2007. Snort#8482; User Manual 2.6.1. Sourcefire, Inc.
• 21: Jacob W. Ulvila , John E. Gaffney, A Decision Analysis Method for Evaluating Computer Intrusion Detection Systems, Decision Analysis, v.1 n.1, p.35-50, March 2004 [doi>10.1287/deca.1030.0001]
• 22: Warrender, C., Forrest, S., and Pearlmutter, B. 1999. Detecting intrusions using system calls: Alternative data models. In Proceedings of the IEEE Symposium on Security and Privacy (SP'99). Oakland, CA, 133--145.
• 23: Zweig, M. H. and Campbell, G. 1993. Receiver-operating characteristic (ROC) plots: A fundamental evaluation tool in clinical medicine. Clin. Chem. 39, 561--577.

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.5 Security and Protection (D.4.6, K.4.2)

Additional Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.6 Security and Protection

General Terms:
Performance, Security

Keywords:
Computer security, intrusion detection, optimal inspection rates, optimal operating points