Exploiting redundancy in question answering

ABSTRACT

Our goal is to automatically answer brief factual questions of the form ``When was the Battle of Hastings?'' or ``Who wrote The Wind in the Willows?''. Since the answer to nearly any such question can now be found somewhere on the Web, the problem reduces to finding potential answers in large volumes of data and validating their accuracy. We apply a method for arbitrary passage retrieval to the first half of the problem and demonstrate that answer redundancy can be used to address the second half. The success of our approach depends on the idea that the volume of available Web data is large enough to supply the answer to most factual questions multiple times and in multiple contexts. A query is generated from a question and this query is used to select short passages that may contain the answer from a large collection of Web data. These passages are analyzed to identify candidate answers. The frequency of these candidates within the passages is used to ``vote'' for the most likely answer. The approach is experimentally tested on questions taken from the TREC-9 question-answering test collection. As an additional demonstration, the approach is extended to answer multiple choice trivia questions of the form typically asked in trivia quizzes and television game shows.

REFERENCES

• 1: Eric Breck, John Burger, David House, Marc Light, and Inderjeet Mani. Question answering from large document collections. In 1999 AAAI Fall Symposium on Question Answering Systems, North Falmouth, MA, 1999.
• 2: Sergey Brin , Lawrence Page, The anatomy of a large-scale hypertextual Web search engine, Proceedings of the seventh international conference on World Wide Web 7, p.107-117, April 1998, Brisbane, Australia
• 3: Claire Cardie. Empirical methods in information extraction. AI Magazine, 18(4):65-79, Winter 1997.
• 4: Claire Cardie , Vincent Ng , David Pierce , Chris Buckley, Examining the role of statistical and linguistic knowledge sources in a general-knowledge question-answering system, Proceedings of the sixth conference on Applied natural language processing, p.180-187, April 29-May 04, 2000, Seattle, Washington [doi>10.3115/974147.974172]
• 5: C. L. A. Clarke, G. V. Cormack, D. I. E. Kisman, and T. R. Lynam. Question answering by passage selection. In 9th Text REtrieval Conference, Gaithersburg, MD, 2000.
• 6: Charles L. A. Clarke , Gordon V. Cormack , Elizabeth A. Tudhope, Relevance ranking for one to three term queries, Information Processing and Management: an International Journal, v.36 n.2, p.291-311, Jan.1.2000 [doi>10.1016/S0306-4573(99)00017-5]
• 7: G. V. Cormack, C. L. A. Clarke, C. R. Palmer, and D. I. E. Kisman. Fast automatic passage ranking. In 8th Text REtrieval Conference, Gaithersburg, MD, November 1999.
• 8: Anne Diekema, Xiaoyong Liu, Jiangping Chen, Hudong Wang, Nancy McCracken, Ozgur Yilmazel, and Elizabeth D. Liddy. Question answering: CNLP at the TREC-9 question answering track. In 9th Text REtrieval Conference, Gaithersburg, MD, 2000.
• 9: Ralph Grishman , Beth Sundheim, Design of the MUC-6 evaluation, Proceedings of the 6th conference on Message understanding, November 06-08, 1995, Columbia, Maryland [doi>10.3115/1072399.1072401]
• 10: Sanda M. Harabagiu and Steven J. Maiorano. Finding answers in large collections of texts: Paragraph indexing + abductive inference. In 1999 AAAI Fall Symposium on Question Answering Systems, pages 63-71, North Falmouth, MA, 1999.
• 11: Sanda M. Harabagiu, Dan Moldovan, Marius Pacsca, Rada Mihalcea, Mihai Surdeanu, Razvan Bunescu, Roxana Girju, Vasile Rus, and Paul Morarescu. FALCON: Boosting knowledge for answer engines. In 9th Text REtrieval Conference, Gaithersburg, MD, 2000.
• 12: David Hawking , Nick Craswell , Paul Thistlewaite , Donna Harman, Results and challenges in Web search evaluation, Proceeding of the eighth international conference on World Wide Web, p.1321-1330, May 1999, Toronto, Canada
• 13: Eduard Hovy, Ulf Hermjakob, Chin-Yew Lin, Mike Junk, and Laurie Gerber. The Webclopedia. In 9th Text REtrieval Conference, Gaithersburg, MD, 2000.
• 14: Abraham Ittycheriah, Martin Franz, Wei-Jing Zhu, and Adwait Ratnaparkhi. IBM's statistical question answering system. In 9th Text REtrieval Conference, Gaithersburg, MD, 2000.
• 15: Julian Kupiec, MURAX: a robust linguistic approach for question answering using an on-line encyclopedia, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.181-190, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States [doi>10.1145/160688.160717]
• 16: John Prager , Eric Brown , Anni Coden , Dragomir Radev, Question-answering by predictive annotation, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.184-191, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345574]
• 17: John Prager, Dragomir R Radev, Eric Brown, Amni Coden, and Valerie Samn. The use of predictive annotation for question answering in TREC-8. In 8th Text REtrieval Conference, Gaithersburg, MD, 1999.
• 18: Dragomir R. Radev , John Prager , Valerie Samn, Ranking suspected answers to natural language questions using predictive annotation, Proceedings of the sixth conference on Applied natural language processing, p.150-157, April 29-May 04, 2000, Seattle, Washington [doi>10.3115/974147.974168]
• 19: Alan F. Smeaton, Information Retrieval: Still Butting Heads with Natural Language Processing?, International Summer School on Information Extraction: A Multidisciplinary Approach to an Emerging Information Technology, p.115-138, January 1997
• 20: Rohini Srihari and Wei Li. Information extraction supported question answering. In 8th Text REtrieval Conference, Gaithersburg, MD, 1999.
• 21: Ellen M. Voorhees and Donna Harman, editors. Proceedings of the Ninth Text REtrieval Conference, Gaithersburg, MD, 2000. See trec.nist.goc.
• 22: Ellen M. Voorhees , Dawn M. Tice, Building a question answering test collection, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.200-207, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345577]
• 23: W. A. Woods, Stephen Green, Paul Martin, and Ann Houston. Halfway to question answering. In 9th Text REtrieval Conference, Gaithersburg, MD, 2000.
• 24: Xiaolan Zhu , Susan Gauch, Incorporating quality metrics in centralized/distributed information retrieval on the World Wide Web, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.288-295, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345602]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Linguistic processing
      H.3.3 Information Search and Retrieval
          Subjects: Search process

General Terms:
Design, Experimentation, Performance