Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

Previous work on Natural Language Processing for Information Retrieval has shown the inadequateness of semantic and syntactic structures for both document retrieval and categorization. The main reason is the high reliability and effectiveness of language models, which are sufficient to accurately solve such retrieval tasks. However, when the latter involve the computation of relational semantics between text fragments simple statistical models may result ineffective. In this paper, we show that syntactic and semantic structures can be used to greatly improve complex categorization tasks such as determining if an answer correctly responds to a question. Given the high complexity of representing semantic/syntactic structures in learning algorithms, we applied kernel methods along with Support Vector Machines to better exploit the needed relational information. Our experiments on answer classification on Web and TREC data show that our models greatly improve on bag-of-words.

REFERENCES

• 1: J. Allan. Natural language processing for information retrieval. In NAACL/ANLP (tutorial notes), 2000.
• 2: Matthew W. Bilotti , Paul Ogilvie , Jamie Callan , Eric Nyberg, Structured retrieval for question answering, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277802]
• 3: S. Blair-Goldensohn, K. R. McKeown, and A. H. Schlaikjer. Answering definitional questions: A hybrid approach. In M. Maybury, editor, New Directions In Question Answering. AAAI Press, 2004.
• 4: Nicola Cancedda , Eric Gaussier , Cyril Goutte , Jean Michel Renders, Word sequence kernels, The Journal of Machine Learning Research, 3, 3/1/2003
• 5: X. Carreras and L. Màrquez. Introduction to the CoNLL-2005 shared task: SRL. In CoNLL, 2005.
• 6: Yi Chen , Ming Zhou , Shilong Wang, Reranking answers for definitional QA using language modeling, Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, p.1081-1088, July 17-18, 2006, Sydney, Australia [doi>10.3115/1220175.1220311]
• 7: Michael Collins , Nigel Duffy, New ranking algorithms for parsing and tagging: kernels over discrete structures, and the voted perceptron, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1073083.1073128]
• 8: Kevyn Collins-Thompson , Jamie Callan , Egidio Terra , Charles L.A. Clarke, The effect of document retrieval quality on factoid question answering performance, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009127]
• 9: Hang Cui , Min-Yen Kan , Tat-Seng Chua, Generic soft pattern models for definitional question answering, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076101]
• 10: Aron Culotta , Jeffrey Sorensen, Dependency tree kernels for relation extraction, Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, p.423-es, July 21-26, 2004, Barcelona, Spain [doi>10.3115/1218955.1219009]
• 11: C. Cumby and D. Roth. Kernel Methods for Relational Learning. In Proceedings of ICML 2003, pages 107--114, Washington, DC, USA, 2003.
• 12: J. Furnkranz, T. Mitchell, and E. Rilof. A case study in using linguistic phrases for text categorization on the www. In Working Notes of the AAAI/ICML, Workshop on Learning for Text Categorization, 1998.
• 13: A. Hickl, J. Williams, J. Bensley, K. Roberts, Y. Shi, and B. Rink. Question answering with LCC's CHAUCER at TREC 2006. In Proceedings of TREC'06, 2006.
• 14: R. Jackendoff. Semantic Structures. MIT Press, 1990.
• 15: Thorsten Joachims, Making large-scale support vector machine learning practical, Advances in kernel methods: support vector learning, MIT Press, Cambridge, MA, 1999
• 16: Christopher Johnson , Charles J. Fillmore, The FrameNet tagset for frame-semantic and syntactic coding of predicate-argument structure, Proceedings of the first conference on North American chapter of the Association for Computational Linguistics, p.56-62, April 29-May 04, 2000, Seattle, Washington
• 17: Jun'ichi Kazama , Kentaro Torisawa, Speeding up training with tree kernels for node relation labeling, Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, p.137-144, October 06-08, 2005, Vancouver, British Columbia, Canada [doi>10.3115/1220575.1220593]
• 18: H. Kazawa, H. Isozaki, and E. Maeda. NTT Question Answering system in TREC 2001. In TREC'01, 2001.
• 19: P. Kingsbury and M. Palmer. From Treebank to PropBank. In LREC'02, 2002.
• 20: Taku Kudo , Yuji Matsumoto, Fast methods for kernel-based text analysis, Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, p.24-31, July 07-12, 2003, Sapporo, Japan [doi>10.3115/1075096.1075100]
• 21: Taku Kudo , Jun Suzuki , Hideki Isozaki, Boosting-based parse reranking with subtree features, Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, p.189-196, June 25-30, 2005, Ann Arbor, Michigan [doi>10.3115/1219840.1219864]
• 22: David D. Lewis, An evaluation of phrasal and clustered representations on a text categorization task, Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, p.37-50, June 21-24, 1992, Copenhagen, Denmark [doi>10.1145/133160.133172]
• 23: Xin Li , Dan Roth, Learning question classifiers: the role of semantic information, Natural Language Engineering, v.12 n.3, p.229-249, September 2006 [doi>10.1017/S1351324905003955]
• 24: M. Surdeanu, M. Ciaramita and H. Zaragoza. Learning to rank answers on large online QA collections. In ACL, 2008.
• 25: A. Moschitti. Efficient convolution kernels for dependency and constituent syntactic trees. In ECML'06, 2006.
• 26: A. Moschitti and R. Basili. Complex linguistic features for text classification: A comprehensive study. In S. McDonald and J. Tait, editors, ECIR, Sunderland, UK, 2004.
• 27: A. Moschitti, B. Coppola, A. Giuglea, and R. Basili. Hierarchical semantic role labeling. In CoNLL 2005 shared task, 2005.
• 28: A. Moschitti, S. Quarteroni, R. Basili, and S. Manandhar. Exploiting syntactic and shallow semantic kernels for question/answer classification. In ACL'07, Prague, Czech Republic, 2007.
• 29: Srini Narayanan , Sanda Harabagiu, Question answering based on semantic structures, Proceedings of the 20th international conference on Computational Linguistics, p.693-es, August 23-27, 2004, Geneva, Switzerland [doi>10.3115/1220355.1220455]
• 30: John Shawe-Taylor , Nello Cristianini, Kernel Methods for Pattern Analysis, Cambridge University Press, New York, NY, 2004
• 31: D. Shen and M. Lapata. Using semantic roles to improve Question Answering. In Proceedings of EMNLP-CoNLL, 2007.
• 32: Libin Shen , Anoop Sarkar , Aravind K. Joshi, Using LTAG based features in parse reranking, Proceedings of the 2003 conference on Empirical methods in natural language processing, p.89-96, July 11, 2003 [doi>10.3115/1119355.1119367]
• 33: Sharon Small , Tomek Strzalkowski , Ting Liu , Sean Ryan , Robert Salkin , Nobuyuki Shimizu , Paul Kantor , Diane Kelly , Robert Rittman , Nina Wacholder, HITIQA: towards analytical question answering, Proceedings of the 20th international conference on Computational Linguistics, p.1291-es, August 23-27, 2004, Geneva, Switzerland [doi>10.3115/1220355.1220544]
• 34: A. F. Smeaton. Using NLP or NLP resources for information retrieval tasks. In T. Strzalkowski, editor, Natural language information retrieval, pages 99--111. Kluwer Academic Publishers, Dordrecht, NL, 1999.
• 35: Tomek Strzalkowski, Natural Language Information Retrieval, Kluwer Academic Publishers, Norwell, MA, 1999
• 36: T. Strzalkowski, G. C. Stein, G. B. Wise, J. P. Carballo, P. Tapanainen, T. Jarvinen, A. Voutilainen, and J. Karlgren. Natural language information retrieval: TREC-7 report. In Text REtrieval Conference, pages 164--173, 1998.
• 37: K. Toutanova, P. Markova, and C. Manning. The Leaf Path Projection View of Parse Trees: Exploring String Kernels for HPSG Parse Selection. In D. Lin and D. Wu, editors, Proceedings of EMNLP 2004, pages 166--173, Barcelona, Spain, July 2004.
• 38: Vladimir N. Vapnik, The nature of statistical learning theory, Springer-Verlag New York, Inc., New York, NY, 1995
• 39: Ellen M. Voorhees, Using WordNet to disambiguate word senses for text retrieval, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.171-180, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States [doi>10.1145/160688.160715]
• 40: Ellen M. Voorhees, Query expansion using lexical-semantic relations, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.61-69, July 03-06, 1994, Dublin, Ireland
• 41: E. M. Voorhees. Overview of the TREC 2001 Question Answering track. In Proceedings of the Thirteenth Text REtreival Conference (TREC 2004), 2004.
• 42: Y. Wu, R. Zhang, X. Hu, and H. Kashioka. Learning unsupervised SVM classifier for answer selection in Web Question Answering . In Proceedings of EMNLP-CoNLL, 2007.
• 43: Dell Zhang , Wee Sun Lee, Question classification using support vector machines, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860443]
• 44: Min Zhang , Jie Zhang , Jian Su, Exploring syntactic features for relation extraction using a convolution tree kernel, Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, p.288-295, June 04-09, 2006, New York, New York [doi>10.3115/1220835.1220872]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Linguistic processing

General Terms:
Algorithms, Experimentation, Performance

Keywords:
kernel methods, natural language processing, question answering, support vector machines, text categorization