Semi-supervised co-training and active learning based approach for multi-view intrusion detection

ABSTRACT

Although there is immense data available from networks and hosts, a very small proportion of this data is labeled due to the cost of obtaining expert labels. This proves to be a significant bottle-neck for developing supervised intrusion detection systems that rely solely on labeled data. In spite of the data being collected from real network environments and hence potentially holding valuable information for intrusion detection, such systems can not exploit the remaining unlabeled data. In this work, we intelligently leverage both labeled and unlabeled data. Also, intrusion detection tasks naturally lend themselves into a multi-view scenario, and can benefit significantly if these multiple views are combined meaningfully. In this paper, we propose a co-training method framework for intrusion detection, which is a semi-supervised learning method and can not only utilize unlabeled data, but can also combine multi-view data. We also employ an active learning framework where statistically ambiguous parts of the unlabeled data are identified, which can then be labeled by an expert. This allows for minimal expert labeling while ensuring that the labels obtained from the expert are most informative. In our experiments, we demonstrate that leveraging the unlabeled data using our proposed method significantly reduces the error rate as compared to using the labeled data alone. In addition, our proposed multi-view method has a lower error rate than using a single view.

REFERENCES

• 1: Avrim Blum , Tom Mitchell, Combining labeled and unlabeled data with co-training, Proceedings of the eleventh annual conference on Computational learning theory, p.92-100, July 24-26, 1998, Madison, Wisconsin, United States [doi>10.1145/279943.279962]
• 2: Stephen Clark , James R. Curran , Miles Osborne, Bootstrapping POS taggers using unlabelled data, Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003, p.49-55, May 31, 2003, Edmonton, Canada [doi>10.3115/1119176.1119183]
• 3: Jeffrey Erman , Anirban Mahanti , Martin Arlitt , Ira Cohen , Carey Williamson, Offline/realtime traffic classification using semi-supervised learning, Performance Evaluation, v.64 n.9-12, p.1194-1213, October, 2007 [doi>10.1016/j.peva.2007.06.014]
• 4: Farnaz Gharibian , Ali A. Ghorbani, Comparative Study of Supervised Machine Learning Techniques for Intrusion Detection, Proceedings of the Fifth Annual Conference on Communication Networks and Services Research, p.350-358, May 14-17, 2007 [doi>10.1109/CNSR.2007.22]
• 5: Ian H. Witten , Eibe Frank, Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems), Morgan Kaufmann Publishers Inc., San Francisco, CA, 2005
• 6: Kayacik H. G., Zincir-Heywood A. N., and Heywood M. I. Selecting Features for Intrusion Detection: A Feature Relevance Analysis on KDD 99 Benchmark. In Proceedings of the International Conference on Privacy, Security, and Trust (PST 2005) (Markham, Ontario, Canada, Oct. 12-14), Association for Computer Machinery Press, Morristown, NJ, 2006, 85--89.
• 7: Svetlana Kiritchenko , Stan Matwin, Email classification with co-training, Proceedings of the 2001 conference of the Centre for Advanced Studies on Collaborative research, p.8, November 05-07, 2001, Toronto, Ontario, Canada
• 8: David D. Lewis , William A. Gale, A sequential algorithm for training text classifiers, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.3-12, July 03-06, 1994, Dublin, Ireland
• 9: Lane, T. A Decision-Theoretic, Semi-Supervised Model for Intrusion Detection. Lane, T. In Maloof, M., ed., Machine learning and data mining for computer security: Methods and applications. London: Springer-Verlag. 2006.
• 10: Srinivas Mukkamala , Andrew H. Sung , Ajith Abraham, Intrusion detection using an ensemble of intelligent paradigms, Journal of Network and Computer Applications, v.28 n.2, p.167-182, April 2005 [doi>10.1016/j.jnca.2004.01.003]
• 11: Ion Muslea , Steven Minton , Craig A. Knoblock, Active + Semi-supervised Learning = Robust Multi-View Learning, Proceedings of the Nineteenth International Conference on Machine Learning, p.435-442, July 08-12, 2002
• 12: Vincent Ng , Claire Cardie, Weakly supervised natural language learning without redundant views, Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, p.94-101, May 27-June 01, 2003, Edmonton, Canada [doi>10.3115/1073445.1073468]
• 13: Nigam, K., McCallum, A., and Mitchell, T. Semi-supervised Text Classification Using EM. In Chapelle, O., Zien, A., and Scholkopf, B. (Eds.) Semi-Supervised Learning. MIT Press: Boston, 2006.
• 14: Animesh Patcha , Jung-Min Park, An overview of anomaly detection techniques: Existing solutions and latest technological trends, Computer Networks: The International Journal of Computer and Telecommunications Networking, v.51 n.12, p.3448-3470, August, 2007 [doi>10.1016/j.comnet.2007.02.001]
• 15: Parikh, D., and Chen, T. Bringing Diverse Classifiers to Common Grounds: dtransform. In proceeding of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) (Las Vegas, Nevada, U.S.A, March 30-April 4), IEEE Computer Society Press, Los Alamitos, California, 2008, 3349--3352.
• 16: Strokes, W. J., and Platt, C. J. Aladin: Active Learning for Statistical Intrusion Detection. In Proceeding of Neural Information Process System Conference 2007 Workshop on Machine Learning in Adversarial Environments for Computer Security MIT Press, Vancouver, Canada, 2007, 12--13.
• 17: Anoop Sarkar, Applying co-training methods to statistical parsing, Second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies 2001, p.1-8, June 01-07, 2001, Pittsburgh, Pennsylvania [doi>10.3115/1073336.1073359]
• 18: University of California Department of Information and Computer Science, KDD Cup 99 Intrusion Detection Dataset Task Description, 1999, URL: http://kdd.ics.uci.edu-/databases/kddcup99/kddcup99.html.
• 19: Xiaojin, Z. Semi-supervised Learning Literature Survey. Technical Report 1530, Department of Computer Sciences, University of Wisconsin, Madison, 2005.
• 20: Zissman, M. 1998/99 DARPA Intrusion Detection Evaluation datasets. MIT Lincoln Laboratory, URL: http://www.ll.mit.edu/IST/ideval/data/data_index.html.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.0 General

General Terms:
Algorithms, Security

Keywords:
active learning, co-training, intrusion detection, multi-view, semi-supervised learning