A knowledge-based search engine powered by wikipedia

ABSTRACT

This paper describes Koru, a new search interface that offers effective domain-independent knowledge-based information retrieval. Koru exhibits an understanding of the topics of both queries and documents. This allows it to (a) expand queries automatically and (b) help guide the user as they evolve their queries interactively. Its understanding is mined from the vast investment of manual effort and judgment that is Wikipedia. We show how this open, constantly evolving encyclopedia can yield inexpensive knowledge structures that are specifically tailored to expose the topics, terminology and semantics of individual document collections. We conducted a detailed user study with 12 participants and 10 topics from the 2005 TREC HARD track, and found that Koru and its underlying knowledge base offers significant advantages over traditional keyword search. It was capable of lending assistance to almost every query issued to it; making their entry more efficient, improving the relevance of the documents they return, and narrowing the gap between expert and novice seekers.

REFERENCES

• 1: Allan, J. (2005) HARD Track overview in TREC 2005 high accuracy retrieval from documents. Proc TREC-2005.
• 2: J. Bhogal , A. Macfarlane , P. Smith, A review of ontology based query expansion, Information Processing and Management: an International Journal, v.43 n.4, p.866-886, July, 2007 [doi>10.1016/j.ipm.2006.09.003]
• 3: Bodo Billerbeck , Justin Zobel, Questioning query expansion: an examination of behaviour and parameters, Proceedings of the 15th Australasian database conference, p.69-76, January 01, 2004, Dunedin, New Zealand
• 4: Richard C. Bodner , Fei Song, Knowledge-Based Approaches to Query Expansion in Information Retrieval, Proceedings of the 11th Biennial Conference of the Canadian Society for Computational Studies of Intelligence on Advances in Artificial Intelligence, p.146-158, May 21-24, 1996
• 5: Dave Crane , Eric Pascarello , Darren James, Ajax in Action, Manning Publications Co., Greenwich, CT, 2005
• 6: James R. Curran , Marc Moens, Improvements in automatic thesaurus extraction, Proceedings of the ACL-02 workshop on Unsupervised lexical acquisition, p.59-66, July 12-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1118627.1118635]
• 7: FAO (1995) Agrovoc Multilingual Agricultural Thesaurus, Food and Agricultural Organization of the United Nations.
• 8: Placing search in context: the concept revisited, ACM Transactions on Information Systems (TOIS), v.20 n.1, p.116-131, January 2002 [doi>10.1145/503104.503110]
• 9: Gabrilovich, E. and Markovitch, S. (2005) Feature Generation for Text Categorization Using World Knowledge. Proc. Int Joint Conf on Artificial Intelligence, pp 1048--1053.
• 10: Gabrilovich, E. and Markovitch, S. (2006) Overcoming the Brittleness bottleneck using Wikipedia: enhancing text categorization with encyclopedic knowledge. Proc. American Association for Artificial Intelligence, pp. 1301--1306.
• 11: Jane Greenberg, Automatic query expansion via lexical-semantic relationships, Journal of the American Society for Information Science and Technology, v.52 n.5, p.402-415, March 2001 [doi>10.1002/1532-2890(2001)9999:9999<::AID-ASI1089>3.3.CO;2-B]
• 12: F. A. Grootjen , Th. P. van der Weide, Conceptual query expansion, Data & Knowledge Engineering, v.56 n.2, p.174-193, February 2006 [doi>10.1016/j.datak.2005.03.006]
• 13: Marti A. Hearst, TileBars: visualization of term distribution information in full text information access, Proceedings of the SIGCHI conference on Human factors in computing systems, p.59-66, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223912]
• 14: Rila Mandala , Takenobu Tokunaga , Hozumi Tanaka, Combining multiple evidence from different types of thesaurus for query expansion, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.191-197, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312677]
• 15: George A. Miller, WordNet: a lexical database for English, Communications of the ACM, v.38 n.11, p.39-41, Nov. 1995 [doi>10.1145/219717.219748]
• 16: Milne, D. and Witten, I. H. (2007) Extracting Corpus Specific Knowledge Bases from Wikipedia, Working Paper 03/2007, Department of Computer Science, University of Waikato.
• 17: David Milne , Olena Medelyan , Ian H. Witten, Mining Domain-Specific Thesauri from Wikipedia: A Case Study, Proceedings of the 2006 IEEE/WIC/ACM International Conference on Web Intelligence, p.442-448, December 18-22, 2006 [doi>10.1109/WI.2006.119]
• 18: Ian Ruthven, Re-examining the potential effectiveness of interactive query expansion, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860475]
• 19: Shiri, A. and Revie, C. (2005) Usability and user perceptions of a thesaurus-enhanced search interface. Journal of Documentation 61(5), 640--656.
• 20: Ali Shiri , Crawford Revie, Query expansion behavior within a thesaurus-enhanced search environment: A user-centered evaluation, Journal of the American Society for Information Science and Technology, v.57 n.4, p.462-478, February 2006 [doi>10.1002/asi.v57:4]
• 21: Ellen M. Voorhees, Query expansion using lexical-semantic relations, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.61-69, July 03-06, 1994, Dublin, Ireland

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Query formulation

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

General Terms:
Algorithms, Design, Experimentation

Keywords:
data mining, information retrieval, query expansion, thesauri, wikipedia