Query dependent pseudo-relevance feedback based on wikipedia

ABSTRACT

Pseudo-relevance feedback (PRF) via query-expansion has been proven to be e®ective in many information retrieval (IR) tasks. In most existing work, the top-ranked documents from an initial search are assumed to be relevant and used for PRF. One problem with this approach is that one or more of the top retrieved documents may be non-relevant, which can introduce noise into the feedback process. Besides, existing methods generally do not take into account the significantly different types of queries that are often entered into an IR system. Intuitively, Wikipedia can be seen as a large, manually edited document collection which could be exploited to improve document retrieval effectiveness within PRF. It is not obvious how we might best utilize information from Wikipedia in PRF, and to date, the potential of Wikipedia for this task has been largely unexplored. In our work, we present a systematic exploration of the utilization of Wikipedia in PRF for query dependent expansion. Specifically, we classify TREC topics into three categories based on Wikipedia: 1) entity queries, 2) ambiguous queries, and 3) broader queries. We propose and study the effectiveness of three methods for expansion term selection, each modeling the Wikipedia based pseudo-relevance information from a different perspective. We incorporate the expansion terms into the original query and use language modeling IR to evaluate these methods. Experiments on four TREC test collections, including the large web collection GOV2, show that retrieval performance of each type of query can be improved. In addition, we demonstrate that the proposed method out-performs the baseline relevance model in terms of precision and robustness.

REFERENCES

• 1: Christopher M. Bishop, Pattern Recognition and Machine Learning (Information Science and Statistics), Springer-Verlag New York, Inc., Secaucus, NJ, 2006
• 2: C. Buckley, G. Salton, J. Allan, and A. Singhal. Automatic query expansion using SMART: TREC 3. In Text REtrieval Conference, 1994.
• 3: Guihong Cao , Jian-Yun Nie , Jianfeng Gao , Stephen Robertson, Selecting good expansion terms for pseudo-relevance feedback, Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, July 20-24, 2008, Singapore, Singapore [doi>10.1145/1390334.1390377]
• 4: C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines. 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
• 5: Paul - Alexandru Chirita , Claudiu S. Firan , Wolfgang Nejdl, Personalized query expansion for the web, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277746]
• 6: Steve Cronen-Townsend , Yun Zhou , W. Bruce Croft, A framework for selective query expansion, Proceedings of the thirteenth ACM international conference on Information and knowledge management, November 08-13, 2004, Washington, D.C., USA [doi>10.1145/1031171.1031220]
• 7: Jonathan L. Elsas , Jaime Arguello , Jamie Callan , Jaime G. Carbonell, Retrieval and feedback models for blog feed search, Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, July 20-24, 2008, Singapore, Singapore [doi>10.1145/1390334.1390394]
• 8: C. Fautsch and J. Savoy. UniNE at TREC 2008: Fact and Opinion Retrieval in the Blogsphere. In Proceedings of TREC 2008.
• 9: Sisay Fissaha Adafre , Valentin Jijkoun , Maarten de Rijke, Fact Discovery in Wikipedia, Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence, p.177-183, November 02-05, 2007 [doi>10.1109/WI.2007.57]
• 10: Bruno M. Fonseca , Paulo Golgher , Bruno Pôssas , Berthier Ribeiro-Neto , Nivio Ziviani, Concept-based interactive query expansion, Proceedings of the 14th ACM international conference on Information and knowledge management, October 31-November 05, 2005, Bremen, Germany [doi>10.1145/1099554.1099726]
• 11: G.R. Giambattista Amati, Claudio Carpineto and F.U. Bordoni. Query di±culty, robustness and selective application of query expansion. In Proceedings of ECIR 2004, pages 127--137, 2004.
• 12: Hang Cui , Ji-Rong Wen , Jian-Yun Nie , Wei-Ying Ma, Query Expansion by Mining User Logs, IEEE Transactions on Knowledge and Data Engineering, v.15 n.4, p.829-839, July 2003 [doi>10.1109/TKDE.2003.1209002]
• 13: Ben He , Iadh Ounis, Combining fields for query expansion and adaptive query expansion, Information Processing and Management: an International Journal, v.43 n.5, p.1294-1307, September, 2007 [doi>10.1016/j.ipm.2006.11.002]
• 14: Indri. http://www.lemurproject.org/indri/.
• 15: John A. Hartigan, Clustering Algorithms, John Wiley & Sons, Inc., New York, NY, 1975
• 16: W. W.-J. H.K. Balog, E. Meij and M. de Rijke. The University of Amsterdam at TREC 2008: Blog, Enterprise, and Relevance Feedback. In Proceedings of TREC 2008.
• 17: K. L. Kwok , M. Chan, Improving two-stage ad-hoc retrieval for short queries, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.250-256, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291003]
• 18: Victor Lavrenko , W. Bruce Croft, Relevance based language models, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.120-127, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383972]
• 19: Kyung Soon Lee , W. Bruce Croft , James Allan, A cluster-based resampling method for pseudo-relevance feedback, Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, July 20-24, 2008, Singapore, Singapore [doi>10.1145/1390334.1390376]
• 20: Yinghao Li , Wing Pong Robert Luk , Kei Shiu Edward Ho , Fu Lai Korris Chung, Improving weak ad-hoc queries using wikipedia asexternal corpus, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277914]
• 21: Donald Metzler , W. Bruce Croft, Latent concept expansion using markov random fields, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277796]
• 22: D. Metzler, T. Strohman, H. Turtle, and W. Croft. Indri at trec 2005: Terabyte track. In Proceedings of TREC 2004.
• 23: David N. Milne , Ian H. Witten , David M. Nichols, A knowledge-based search engine powered by wikipedia, Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, November 06-10, 2007, Lisbon, Portugal [doi>10.1145/1321440.1321504]
• 24: G. Mishne. Applied Text Analytics for Blogs. PhD thesis, University of Amsterdam, Amsterdam, 2007.
• 25: J. Platt. Probabilities for SV machines. Advances in large margin classifiers, pages 61--74.
• 26: Stephen Robertson , Hugo Zaragoza , Michael Taylor, Simple BM25 extension to multiple weighted fields, Proceedings of the thirteenth ACM international conference on Information and knowledge management, November 08-13, 2004, Washington, D.C., USA [doi>10.1145/1031171.1031181]
• 27: S.E. Robertson, S. Walker, M. Beaulieu, M. Gatford, and A. Payne. Okapi at TREC-4. In In Proceedings of the 4th Text REtrieval Conference (TREC), 1996.
• 28: Mark Sanderson, Ambiguous queries: test collections need more sense, Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, July 20-24, 2008, Singapore, Singapore [doi>10.1145/1390334.1390420]
• 29: Tao Tao , ChengXiang Zhai, Regularized estimation of mixture models for robust pseudo-relevance feedback, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA [doi>10.1145/1148170.1148201]
• 30: Jinxi Xu , W. Bruce Croft, Improving the effectiveness of information retrieval with local context analysis, ACM Transactions on Information Systems (TOIS), v.18 n.1, p.79-112, Jan. 2000 [doi>10.1145/333135.333138]
• 31: Elad Yom-Tov , Shai Fine , David Carmel , Adam Darlow, Learning to estimate query difficulty: including applications to missing content detection and distributed information retrieval, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076121]
• 32: M.M. Zesch Torsten, Gurevych Iryna. Analyzing and accessing Wikipedia as a lexical semantic resource. In Biannual Conference of the Society for Computational Linguistics and Language Technology 2007, pages 213--221.
• 33: Chengxiang Zhai , John Lafferty, Model-based feedback in the language modeling approach to information retrieval, Proceedings of the tenth international conference on Information and knowledge management, October 05-10, 2001, Atlanta, Georgia, USA [doi>10.1145/502585.502654]
• 34: W. Zhang and C. Yu. UIC at TREC 2006 Blog Track. In The Fifteenth Text REtrieval Conference (TREC 2006) Proceedings, 2007.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Design, Experimentation, Performance

Keywords:
entity, information retrieval, pseudo-relevance feedback, query expansion, wikipedia