Principled reasoning and practical applications of alert fusion in intrusion detection systems

ABSTRACT

It is generally believed that by combining several diverse intrusion detectors (i.e., forming an IDS ensemble), we may achieve better performance. However, there has been very little work on analyzing the effectiveness of an IDS ensemble. In this paper, we study the following problem: how to make a good fusion decision on the alerts from multiple detectors in order to improve the final performance. We propose a decision-theoretic alert fusion technique based on the likelihood ratio test (LRT). We report our experience from empirical studies, and formally analyze its practical interpretation based on ROC curve analysis. Through theoretical reasoning and experiments using multiple IDSs on several data sets, we show that our technique is more flexible and also outperforms other existing fusion techniques such as AND, OR, majority voting, and weighted voting.

REFERENCES

• 1: Kdd cup 1999 data. http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html, 2005.
• 2: Nahla Ben Amor , Salem Benferhat , Zied Elouedi, Naive Bayes vs decision trees in intrusion detection systems, Proceedings of the 2004 ACM symposium on Applied computing, March 14-17, 2004, Nicosia, Cyprus [doi>10.1145/967900.967989]
• 3: Ashish Arora , Dennis Hall , C. Ariel Pinto , Dwayne Ramsey , Rahul Telang, Measuring the Risk-Based Value of IT Security Solutions, IT Professional, v.6 n.6, p.35-42, November 2004 [doi>10.1109/MITP.2004.89]
• 4: Stefan Axelsson, The base-rate fallacy and its implications for the difficulty of intrusion detection, Proceedings of the 6th ACM conference on Computer and communications security, p.1-7, November 01-04, 1999, Kent Ridge Digital Labs, Singapore [doi>10.1145/319709.319710]
• 5: Marco Barreno, Alvaro A. Cardenas, and J. D. Tygar. Optimal roc curve for a combination of classifiers. In Proceedings of Neural Information Processing Systems (NIPS) 20, 2008.
• 6: Tim Bass, Intrusion detection systems and multisensor data fusion, Communications of the ACM, v.43 n.4, p.99-105, April 2000 [doi>10.1145/332051.332079]
• 7: J. De Bonet, C. Isbell, and P. Viola. Mimic: Finding optima by estimating probability densities. Advances in Neural Information Processing Systems, 9, 1997.
• 8: Leo Breiman, Bagging predictors, Machine Learning, v.24 n.2, p.123-140, Aug. 1996 [doi>10.1023/A:1018054314350]
• 9: Alvaro A. Cárdenas , John S. Baras , Karl Seamon, A Framework for the Evaluation of Intrusion Detection Systems, Proceedings of the 2006 IEEE Symposium on Security and Privacy, p.63-77, May 21-24, 2006 [doi>10.1109/SP.2006.2]
• 10: Chih-Chung Chang and Chih-Jen Lin. LIBSVM: a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
• 11: Frédéric Cuppens , Alexandre Miège, Alert Correlation in a Cooperative Intrusion Detection Framework, Proceedings of the 2002 IEEE Symposium on Security and Privacy, p.202, May 12-15, 2002
• 12: Hervé Debar , Andreas Wespi, Aggregation and Correlation of Intrusion-Detection Alerts, Proceedings of the 4th International Symposium on Recent Advances in Intrusion Detection, p.85-103, October 10-12, 2001
• 13: Luca Didaci, Giorgio Giacinto, and Fabio Roli. Ensemble learning for intrusion detection in computer networks. http://citeseer.ist.psu.edu/533620.html.
• 14: Thomas G. Dietterich, Ensemble Methods in Machine Learning, Proceedings of the First International Workshop on Multiple Classifier Systems, p.1-15, June 21-23, 2000
• 15: Wei Fan , Wenke Lee , Salvatore J. Stolfo , Matthew Miller, A Multiple Model Cost-Sensitive Approach for Intrusion Detection, Proceedings of the 11th European Conference on Machine Learning, p.142-153, May 31-June 02, 2000
• 16: Wei Fan , Salvatore J. Stolfo , Junxin Zhang , Philip K. Chan, AdaCost: Misclassification Cost-Sensitive Boosting, Proceedings of the Sixteenth International Conference on Machine Learning, p.97-105, June 27-30, 1999
• 17: Y. Freund and R. E. Schapire. Experiments with a new boosting algorithm. In Thirteenth International Conference on Machine Learning (ICML), pages 148--156, 1996.
• 18: G. Giacinto and F. Roli. Intrusion detection in computer networks by multiple classifier systems. In Proceedings of 16th International Conference on Pattern Recognition (ICPR 2002), 2002.
• 19: Guofei Gu , Prahlad Fogla , David Dagon , Wenke Lee , Boris Skorić, Measuring intrusion detection capability: an information-theoretic approach, Proceedings of the 2006 ACM Symposium on Information, computer and communications security, March 21-24, 2006, Taipei, Taiwan [doi>10.1145/1128817.1128834]
• 20: Guofei Gu, Prahlad Fogla, David Dagon, Wenke Lee, and Boris Skoric. Towards an information-theoretic framework for analyzing intrusion detection systems. In Proceedings of the 11th European Symposium on Research in Computer Security (ESORICS'06), September 2006.
• 21: Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning. Springer-Verlag New York, Inc., 2003.
• 22: Imad Y. Hoballah and Pramod K. Varshney. Distributed Bayesian signal detection. IEEE Transactions on Information Theory, 35(5):995--1000, 1989.
• 23: Wenjie Hu, Yihua Liao, and V. Rao Vemuri. Robust support vector machines for anomaly detection in computer security. In Proc. 2003 International Conference on Machine Learning and Applications (ICMLA'03), 2003.
• 24: Finn V. Jensen, Bayesian Networks and Decision Graphs, Springer-Verlag New York, Inc., Secaucus, NJ, 2001
• 25: Michael I. Jordan, Learning in graphical models, MIT Press, Cambridge, MA, 1999
• 26: Christopher Kruegel , Giovanni Vigna, Anomaly detection of web-based attacks, Proceedings of the 10th ACM conference on Computer and communications security, October 27-30, 2003, Washington D.C., USA [doi>10.1145/948109.948144]
• 27: Christopher Kruegel , Giovanni Vigna , William Robertson, A multi-model approach to the detection of web-based attacks, Computer Networks: The International Journal of Computer and Telecommunications Networking, v.48 n.5, p.717-738, 5 August 2005 [doi>10.1016/j.comnet.2005.01.009]
• 28: Christopher Kruegel , Darren Mutz , William Robertson , Fredrik Valeur, Bayesian Event Classification for Intrusion Detection, Proceedings of the 19th Annual Computer Security Applications Conference, p.14, December 08-12, 2003
• 29: Ludmila I. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms, Wiley-Interscience, 2004
• 30: Wenke Lee , Wei Fan , Matthew Miller , Salvatore J. Stolfo , Erez Zadok, Toward cost-sensitive modeling for intrusion detection and response, Journal of Computer Security, v.10 n.1-2, p.5-22, 2002
• 31: Wenke Lee , Salvatore J. Stolfo, A framework for constructing features and models for intrusion detection systems, ACM Transactions on Information and System Security (TISSEC), v.3 n.4, p.227-261, Nov. 2000 [doi>10.1145/382912.382914]
• 32: Yihua Liao , V. Rao Vemuri, Using Text Categorization Techniques for Intrusion Detection, Proceedings of the 11th USENIX Security Symposium, p.51-59, August 05-09, 2002
• 33: R. P. Lippmann, D. J. Fried, I. Graf, J. W. Haines, K. P. Kendall, D. McClung, D. Weber, S. E. Webster, D. Wyschogrod, R. K. Cunningham, and M. A. Zissman. Evaluating intrusion detection systems: The 1998 darpa off-line intrusion detection evaluation. In Proceedings of the 2000 DARPA Information Survivability Conference and Exposition (DISCEX'00), 2000.
• 34: Matthew V. Mahoney, Network traffic anomaly detection based on packet bytes, Proceedings of the 2003 ACM symposium on Applied computing, March 09-12, 2003, Melbourne, Florida [doi>10.1145/952532.952601]
• 35: M. Mahoney and P. Chan. An analysis of the 1999 darpa/lincoln laboratory evaluation data for network anomaly detection. In Proceedings of the 6th International Symposium on Recent Advances in Intrusion Detection (RAID'03), 2003.
• 36: Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory, ACM Transactions on Information and System Security (TISSEC), v.3 n.4, p.262-294, Nov. 2000 [doi>10.1145/382912.382923]
• 37: Thomas M. Mitchell, Machine Learning, McGraw-Hill Higher Education, 1997
• 38: J. Neyman and E. S. Pearson. On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London, Series A, Containing Papers of a Mathematical or Physical Character, 231:289--337, 1933.
• 39: Peng Ning , Yun Cui , Douglas S. Reeves, Constructing attack scenarios through correlation of intrusion alerts, Proceedings of the 9th ACM conference on Computer and communications security, November 18-22, 2002, Washington, DC, USA [doi>10.1145/586110.586144]
• 40: Roberto Perdisci , Guofei Gu , Wenke Lee, Using an Ensemble of One-Class SVM Classifiers to Harden Payload-based Anomaly Detection Systems, Proceedings of the Sixth International Conference on Data Mining, p.488-498, December 18-22, 2006 [doi>10.1109/ICDM.2006.165]
• 41: Phillip A. Porras, Martin W. Fong, and Alfonso Valdes. A mission-impact-based approach to infosec alarm correlation. In Proceedings of the 5th International Symposium on Recent Advances in Intrusion Detection (RAID'02), 2002.
• 42: Rain Forest Puppy. Libwhisker official release v2.1, 2004. Available at http://www.wiretrip.net/rfp/lw.asp.
• 43: Martin Roesch, Snort - Lightweight Intrusion Detection for Networks, Proceedings of the 13th USENIX conference on System administration, November 07-12, 1999, Seattle, Washington
• 44: M. Shankar, N. Rao, and S. Batsell. Fusing intrusion data for detection and containment. In Proceedings of MILCOM2003, 2003.
• 45: Sal Stolfo, Wei Fan, Wenke Lee, Andreas Prodromidis, and Phil Chan. Cost-based modeling for fraud and intrusion detection: Results from the jam project. In Proceedings of the 2000 DARPA Information Survivability Conference and Exposition (DISCEX '00), 2000.
• 46: Eric Totel, Frederic Majorczyk, and Ludovic Me. COTS diversity intrusion detection and application to web servers. In Proceedings of RAID'2005, September 2005.
• 47: Fredrik Valeur , Giovanni Vigna , Christopher Kruegel , Richard A. Kemmerer, A Comprehensive Approach to Intrusion Detection Alert Correlation, IEEE Transactions on Dependable and Secure Computing, v.1 n.3, p.146-169, July 2004 [doi>10.1109/TDSC.2004.21]
• 48: Vladimir N. Vapnik, The nature of statistical learning theory, Springer-Verlag New York, Inc., New York, NY, 1995
• 49: Pramod K. Varshney, Distributed Detection and Data Fusion, Springer-Verlag New York, Inc., Secaucus, NJ, 1996
• 50: Ke Wang and Salvatore J. Stolfo. Anomalous payload-based network intrusion detection. In Proceedings of RAID'2004, September 2004.
• 51: David H. Wolpert, Stacked generalization, Neural Networks, v.5 n.2, p.241-259, 1992 [doi>10.1016/S0893-6080(05)80023-1]
• 52: L. Xu, A. Krzyzak, and CY Suen. Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans. Systems Man Cybernet, 22(3):418--435, 1992.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.0 General

Additional Classification:
  B. Hardware
  B.8 Performance and Reliability
      B.8.2 Performance Analysis and Design Aids

General Terms:
Security

Keywords:
IDS ensemble, ROC curve, alert fusion, intrusion detection, likelihood ratio test