Ranking community answers by modeling question-answer relationships via analogical reasoning

ABSTRACT

The method of finding high-quality answers has significant impact on user satisfaction in community question answering systems. However, due to the lexical gap between questions and answers as well as spam typically existing in user-generated content, filtering and ranking answers is very challenging. Previous solutions mainly focus on generating redundant features, or finding textual clues using machine learning techniques; none of them ever consider questions and their answers as relational data but instead model them as independent information. Moreover, they only consider the answers of the current question, and ignore any previous knowledge that would be helpful to bridge the lexical and semantic gap. We assume that answers are connected to their questions with various types of latent links, i.e. positive indicating high-quality answers, negative links indicating incorrect answers or user-generated spam, and propose an analogical reasoning-based approach which measures the analogy between the new question-answer linkages and those of relevant knowledge which contains only positive links; the candidate answer which has the most analogous link is assumed to be the best answer. We conducted experiments based on 29.8 million Yahoo!Answer question-answer threads and showed the effectiveness of our approach.

REFERENCES

• 1: Eugene Agichtein , Carlos Castillo , Debora Donato , Aristides Gionis , Gilad Mishne, Finding high-quality content in social media, Proceedings of the international conference on Web search and web data mining, February 11-12, 2008, Palo Alto, California, USA [doi>10.1145/1341531.1341557]
• 2: Jiang Bian , Yandong Liu , Eugene Agichtein , Hongyuan Zha, A few bad votes too many?: towards robust ranking in social media, Proceedings of the 4th international workshop on Adversarial information retrieval on the web, April 22-22, 2008, Beijing, China [doi>10.1145/1451983.1451997]
• 3: Jiang Bian , Yandong Liu , Eugene Agichtein , Hongyuan Zha, Finding the right facts in the crowd: factoid question answering over social media, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367561]
• 4: Matthew W. Bilotti , Paul Ogilvie , Jamie Callan , Eric Nyberg, Structured retrieval for question answering, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277802]
• 5: Mohan John Blooma , Alton Y. K. Chua , Dion Hoe-Lian Goh, A predictive framework for retrieving the best answer, Proceedings of the 2008 ACM symposium on Applied computing, March 16-20, 2008, Fortaleza, Ceara, Brazil [doi>10.1145/1363686.1363944]
• 6: E. Brill, J. Lin, M. Banko, and etc. Data-intensive question answeringa. In TREC, 2001.
• 7: Jennifer Chu-Carroll , Krzysztof Czuba , John Prager , Abraham Ittycheriah, In question answering, two heads are better than one, Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, p.24-31, May 27-June 01, 2003, Edmonton, Canada [doi>10.3115/1073445.1073449]
• 8: Susan Dumais , Michele Banko , Eric Brill , Jimmy Lin , Andrew Ng, Web question answering: is more always better?, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564428]
• 9: R. French. The computational modeling of analogy-marking. Trends in cognitive Sciences, 6, 2002.
• 10: Lisa Getoor , Nir Friedman , Daphne Koller , Benjamin Taskar, Learning probabilistic models of link structure, The Journal of Machine Learning Research, 3, 3/1/2003
• 11: L. Getoor, N. Friedman, and etc. Probabilistic relational models. Introduction to Statistical Relational Learning, 2007.
• 12: Z. Ghahramani and K. Heller. Bayesian sets. In Proc. of NIPS, 2005.
• 13: Tommi S. Jaakkola , Michael I. Jordan, Bayesian parameter estimation via variational methods, Statistics and Computing, v.10 n.1, p.25-37, January 2000 [doi>10.1023/A:1008932416310]
• 14: Jiwoon Jeon , W. Bruce Croft , Joon Ho Lee , Soyeon Park, A framework to predict the quality of answers with non-textual features, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA [doi>10.1145/1148170.1148212]
• 15: Jiwoon Jeon , W. Bruce Croft , Joon Ho Lee, Finding similar questions in large question and answer archives, Proceedings of the 14th ACM international conference on Information and knowledge management, October 31-November 05, 2005, Bremen, Germany [doi>10.1145/1099554.1099572]
• 16: Valentin Jijkoun , Maarten de Rijke, Retrieving answers from frequently asked questions pages on the web, Proceedings of the 14th ACM international conference on Information and knowledge management, October 31-November 05, 2005, Bremen, Germany [doi>10.1145/1099554.1099571]
• 17: Pawel Jurczyk , Eugene Agichtein, Discovering authorities in question answer communities by using link analysis, Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, November 06-10, 2007, Lisbon, Portugal [doi>10.1145/1321440.1321575]
• 18: Jon M. Kleinberg, Authoritative sources in a hyperlinked environment, Journal of the ACM (JACM), v.46 n.5, p.604-632, Sept. 1999 [doi>10.1145/324133.324140]
• 19: J. Ko, L. Si, and E. Nyberg. A probabilistic framework for answer selection in question answering. In Proc. of NAACL/HLT, 2007.
• 20: Jeongwoo Ko , Eric Nyberg , Luo Si, A probabilistic graphical model for joint answer ranking in question answering, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277801]
• 21: J. Leibenluft. Librarianąŕs worst nightmare: Yahoo!answers, where 120 million users can be wrong. Slate Magazine, 2007.
• 22: Jimmy Lin , Boris Katz, Question answering from the web using knowledge annotation and knowledge mining techniques, Proceedings of the twelfth international conference on Information and knowledge management, November 03-08, 2003, New Orleans, LA, USA [doi>10.1145/956863.956886]
• 23: Yandong Liu , Jiang Bian , Eugene Agichtein, Predicting information seeker satisfaction in community question answering, Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, July 20-24, 2008, Singapore, Singapore [doi>10.1145/1390334.1390417]
• 24: Bernardo Magnini , Matteo Negri , Roberto Prevete , Hristo Tanev, Is it the right answer?: exploiting web redundancy for Answer Validation, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1073083.1073154]
• 25: D. Molla and J. Vicedo. Question answering in restricted domains: An overview. In Proc. of ACL, 2007.
• 26: R. Silva, E. Airoldi, and K. Heller. Small sets of interacting proteins suggest latent linkage mechanisms through analogical reasoning. In Gatsby Technical Report, GCNU TR 2007-001, 2007.
• 27: R. Silva, K. Heller, and Z. Ghahramani. Analogical reasoning with relational bayesian sets. In Proc. of AISTATS, 2007.
• 28: Qi Su , Dmitry Pavlov , Jyh-Herng Chow , Wendell C. Baker, Internet-scale collection of human-reviewed data, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242604]
• 29: Xiaobing Xue , Jiwoon Jeon , W. Bruce Croft, Retrieval models for question and answer archives, Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, July 20-24, 2008, Singapore, Singapore [doi>10.1145/1390334.1390416]
• 30: Jun Zhang , Mark S. Ackerman , Lada Adamic, Expertise networks in online communities: structure and algorithms, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242603]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.5 On-line Information Services
          Subjects: Web-based services

Additional Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Correlation and regression analysis

General Terms:
Algorithms, Performance

Keywords:
analogical reasoning, community question answering, probabilistic relational modeling, ranking