Building semantic kernels for text classification using wikipedia

ABSTRACT

Document classification presents difficult challenges due to the sparsity and the high dimensionality of text data, and to the complex semantics of the natural language. The traditional document representation is a word-based vector (Bag of Words, or BOW), where each dimension is associated with a term of the dictionary containing all the words that appear in the corpus. Although simple and commonly used, this representation has several limitations. It is essential to embed semantic information and conceptual patterns in order to enhance the prediction capabilities of classification algorithms. In this paper, we overcome the shortages of the BOW approach by embedding background knowledge derived from Wikipedia into a semantic kernel, which is then used to enrich the representation of documents. Our empirical evaluation with real data sets demonstrates that our approach successfully achieves improved classification accuracy with respect to the BOW technique, and to other recently developed methods.

REFERENCES

• 1: L. AlSumait and C. Domeniconi. Local Semantic Kernels for Text Document Clustering. In Workshop on Text Mining, SIAM International Conference on Data Mining, Minneapolis, MN, 2007. SIAM.
• 2: R. Bunescu and M. Pasca. Using encyclopedic knowledge for named entity disambiguation. In Conference of the European Chapter of the Association for Computational Linguistics, Trento, Italy, 2006.
• 3: Carnegie Group, Inc. and Reuters, Ltd. Reuters-21578 text categorization test collection, 1997.
• 4: C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines, 2001.
• 5: Kushal Dave , Steve Lawrence , David M. Pennock, Mining the peanut gallery: opinion extraction and semantic classification of product reviews, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775226]
• 6: M. de Buenega Rodriguez, J. M. Gomez-Hidalgo, and B. Diaz-Agudo. Using wordnet to complement training information in text categorization. In International Conference on Recent Advances in Natural Language Processing, 1997.
• 7: Ludovic Denoyer , Patrick Gallinari, The Wikipedia XML corpus, ACM SIGIR Forum, v.40 n.1, June 2006 [doi>10.1145/1147197.1147210]
• 8: E. Gabrilovich and S. Markovitch. Feature generation for text categorization using world knowledge. In International Joint Conference on Artificial Intelligence, Edinburgh, Scotland, 2005.
• 9: E. Gabrilovich and S. Markovitch. Overcoming the brittleness bottleneck using wikipedia: enhancing text categorization with encyclopedic knowledge. In National Conference on Artificial Intelligence (AAAI), Boston, Massachusetts, 2006.
• 10: William Hersh , Chris Buckley , T. J. Leone , David Hickam, OHSUMED: an interactive retrieval evaluation and new large test collection for research, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.192-201, July 03-06, 1994, Dublin, Ireland
• 11: A. Hotho, S. Staab, and G. Stumme. Wordnet improves text document clustering. In Semantic Web Workshop, SIGIR Conference, Toronto, Canada, 2003. ACM.
• 12: L. Jing, L. Zhou, M. K. Ng, and J. Z. Huang. Ontology-based distance measure for text clustering. In Workshop on Text Mining, SIAM International Conference on Data Mining, Bethesda, MD, 2006. SIAM.
• 13: Thorsten Joachims, Text Categorization with Suport Vector Machines: Learning with Many Relevant Features, Proceedings of the 10th European Conference on Machine Learning, p.137-142, April 21-23, 1998
• 14: K. Lang. Newsweeder: Learning to filter netnews. In International Conference on Machine Learning, Tahoe City, California, 1995. Morgan Kaufmann.
• 15: David Milne , Olena Medelyan , Ian H. Witten, Mining Domain-Specific Thesauri from Wikipedia: A Case Study, Proceedings of the 2006 IEEE/WIC/ACM International Conference on Web Intelligence, p.442-448, December 18-22, 2006 [doi>10.1109/WI.2006.119]
• 16: Bo Pang , Lillian Lee , Shivakumar Vaithyanathan, Thumbs up?: sentiment classification using machine learning techniques, Proceedings of the ACL-02 conference on Empirical methods in natural language processing, p.79-86, July 06, 2002 [doi>10.3115/1118693.1118704]
• 17: Nello Cristianini , John Shawe-Taylor, An introduction to support Vector Machines: and other kernel-based learning methods, Cambridge University Press, New York, NY, 1999
• 18: John Shawe-Taylor , Nello Cristianini, Kernel Methods for Pattern Analysis, Cambridge University Press, New York, NY, 2004
• 19: Georges Siolas , Florence d'Alché-Buc, Support Vector Machines Based on a Semantic Kernel for Text Categorization, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks (IJCNN'00)-Volume 5, p.5205, July 24-27, 2000
• 20: L. A. Urena-Lopez, M. Buenaga, and J. M. Gomez. Integrating linguistic resources in TC through WSD. Computers and the Humanities, 35:215--230, 2001.
• 21: Pu Wang , Jian Hu , Hua-Jun Zeng , Lijun Chen , Zheng Chen, Improving Text Classification by Using Encyclopedia Knowledge, Proceedings of the 2007 Seventh IEEE International Conference on Data Mining, p.332-341, October 28-31, 2007 [doi>10.1109/ICDM.2007.77]
• 22: S. K. M. Wong , Wojciech Ziarko , Patrick C. N. Wong, Generalized vector spaces model in information retrieval, Proceedings of the 8th annual international ACM SIGIR conference on Research and development in information retrieval, p.18-25, June 05-07, 1985, Montreal, Quebec, Canada [doi>10.1145/253495.253506]
• 23: Yiming Yang , Jan O. Pedersen, A Comparative Study on Feature Selection in Text Categorization, Proceedings of the Fourteenth International Conference on Machine Learning, p.412-420, July 08-12, 1997

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.3 Clustering
          Subjects: Similarity measures

Additional Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.3 Clustering
          Subjects: Algorithms

General Terms:
Algorithms

Keywords:
kernel methods, semantic kernels, text classification, wikipedia