A syntactic tree matching approach to finding similar questions in community-based qa services

ABSTRACT

While traditional question answering (QA) systems tailored to the TREC QA task work relatively well for simple questions, they do not suffice to answer real world questions. The community-based QA systems offer this service well, as they contain large archives of such questions where manually crafted answers are directly available. However, finding similar questions in the QA archive is not trivial. In this paper, we propose a new retrieval framework based on syntactic tree structure to tackle the similar question matching problem. We build a ground-truth set from Yahoo! Answers, and experimental results show that our method outperforms traditional bag-of-word or tree kernel based methods by 8.3% in mean average precision. It further achieves up to 50% improvement by incorporating semantic features as well as matching of potential answers. Our model does not rely on training, and it is demonstrated to be robust against grammatical errors as well.

REFERENCES

• 1: Trec proceedings. http://trec.nist.gov/proceedings/proceedings.html.
• 2: Jiang Bian , Yandong Liu , Eugene Agichtein , Hongyuan Zha, Finding the right facts in the crowd: factoid question answering over social media, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367561]
• 3: Stephan Bloehdorn , Alessandro Moschitti, Structure and semantics for expressive text kernels, Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, November 06-10, 2007, Lisbon, Portugal [doi>10.1145/1321440.1321561]
• 4: R. D. Burke, K. J. Hammond, V. A. Kulyukin, S. L. Lytinen, N. Tomuro, and S. Schoenberg. Question answering from frequently asked question files: Experiences with the faq finder system. AI Magazine, 18(2):57--66, 1997.
• 5: M. Collins and N. Duffy. Convolution kernels for natural language. In Advances in Neural Information Processing Systems 14, pages 625--632. MIT Press, 2001.
• 6: Hang Cui , Renxu Sun , Keya Li , Min-Yen Kan , Tat-Seng Chua, Question answering passage retrieval using dependency relations, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076103]
• 7: A. Diekema, X. Liu, J. Chen, H.Wang, N. Mccracken, O. Yilmazel, and E. D. Liddy. Question answering: Cnlp at the trec-9 question answering track. In Proceedings of the Ninth Text REtrieval Conference (TREC-9), pages 501--510. Department of Commerce, National Institute of Standards and Technology, 2000.
• 8: Jianfeng Gao , Jian-Yun Nie , Guangyuan Wu , Guihong Cao, Dependence language model for information retrieval, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009024]
• 9: Jiwoon Jeon , W. Bruce Croft , Joon Ho Lee, Finding similar questions in large question and answer archives, Proceedings of the 14th ACM international conference on Information and knowledge management, October 31-November 05, 2005, Bremen, Germany [doi>10.1145/1099554.1099572]
• 10: Valentin Jijkoun , Maarten de Rijke, Retrieving answers from frequently asked questions pages on the web, Proceedings of the 14th ACM international conference on Information and knowledge management, October 31-November 05, 2005, Bremen, Germany [doi>10.1145/1099554.1099571]
• 11: Yu-Sheng Lai , Kuao-Ann Fung , Chung-Hsien Wu, FAQ mining via list detection, proceedings of the 2002 conference on multilingual summarization and question answering, p.1-7, August 31, 2002 [doi>10.3115/1118845.1118849]
• 12: C. Leacock and M. Chodrow. Combining local context and WordNet similarity for word sense identification. In WordNet: An Electronic Lexical Database. MIT Press, 1998.
• 13: A. Moschitti. Efficient convolution kernels for dependency and constituent syntactic trees. In ECML, pages 318--329. Springer, 2006.
• 14: A. Moschitti, S. Quarteroni, R. Basili, and S. Manandhar. Exploiting syntactic and shallow semantic kernels for question answer classification. In ACL. The Association for Computer Linguistics, 2007.
• 15: S. Riezler, A. Vasserman, I. Tsochantaridis, V. O. Mittal, and Y. Liu. Statistical machine translation for query expansion in answer retrieval. In ACL. The Association for Computer Linguistics, 2007.
• 16: Fei Song , W. Bruce Croft, A general language model for information retrieval, Proceedings of the eighth international conference on Information and knowledge management, p.316-321, November 02-06, 1999, Kansas City, Missouri, United States [doi>10.1145/319950.320022]
• 17: R. Soricut and E. Brill. Automatic question answering: Beyond the factoid. In HLT-NAACL, pages 57--64, 2004.
• 18: Xiaobing Xue , Jiwoon Jeon , W. Bruce Croft, Retrieval models for question and answer archives, Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, July 20-24, 2008, Singapore, Singapore [doi>10.1145/1390334.1390416]
• 19: D. Zhang and W. S. Lee. Question classification using support vector machines. In SIGIR '03, pages 26--32. ACM, 2003.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Text analysis

General Terms:
Algorithms, Experimentation, Measurement

Keywords:
Yahoo! answer, question answering, question matching, syntactic structure