Document language models, query models, and risk minimization for information retrieval

ABSTRACT

We present a framework for information retrieval that combines document models and query models using a probabilistic ranking function based on Bayesian decision theory. The framework suggests an operational retrieval model that extends recent developments in the language modeling approach to information retrieval. A language model for each document is estimated, as well as a language model for each query, and the retrieval problem is cast in terms of risk minimization. The query language model can be exploited to model user preferences, the context of a query, synonomy and word senses. While recent work has incorporated word translation models for this purpose, we introduce a new method using Markov chains defined on a set of documents to estimate the query models. The Markov chain method has connections to algorithms from link analysis and social networks. The new approach is evaluated on TREC collections and compared to the basic language modeling approach and vector space models together with query expansion using Rocchio. Significant improvements are obtained over standard query expansion methods for strong baseline TF-IDF systems, with the greatest improvements attained for short queries on Web data.

REFERENCES

• 1: Adam Berger , John Lafferty, Information retrieval as statistical translation, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.222-229, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312681]
• 2: A.Bookstein and D.Swanson.A decision theoretic foundation for indexing.Journal for the American Society for Information Science pages 45 -50,1975.
• 3: A.Bookstein and D.Swanson.Probabilistic models for automatic indexing.Journal for the American Society for Information Science 25(5):312 -318,1976.
• 4: Sergey Brin , Lawrence Page, The anatomy of a large-scale hypertextual Web search engine, Proceedings of the seventh international conference on World Wide Web 7, p.107-117, April 1998, Brisbane, Australia
• 5: H.Hubbell C.yAn input-output approach to clique identification.Sociometry 28:377 -399,1965.
• 6: J.G.Carbonell,Y.Geng,and J.Goldstein.Automated query-relevant summarization and diversity-based reranking. In IJCAI-97 Workshop on AI and Digital Libraries 1997.
• 7: W. S. Cooper , M. E. Maron, Foundations of Probabilistic and Utility-Theoretic Indexing, Journal of the ACM (JACM), v.25 n.1, p.67-80, Jan. 1978 [doi>10.1145/322047.322053]
• 8: W.B.Croft and D.J.Harper.Using probabilistic models of document retrieval without relevance information.Journal of Documentation 35:285 -295,1979.
• 9: S.Deerwester,S.Dumais,T.Landauer,G.Furnas,and R.Harshman.Indexing by latent semantic analysis.Journal of American Society for Information Science 41:391 -407, 1990.
• 10: Norbert Fuhr, Probabilistic models in information retrieval, The Computer Journal, v.35 n.3, p.243-255, June 1992 [doi>10.1093/comjnl/35.3.243]
• 11: D.Hiemstra and W.Kraaij.Twenty-one at TREC-7:Ad-hoc and cross-language track.In Proc. of Seventh Text REtrieval Conference (TREC-7),1998.
• 12: L.Katz.A new status index derived from sociometric analysis.Psychometrika 18:39 -43,1953.
• 13: Jon M. Kleinberg, Authoritative sources in a hyperlinked environment, Journal of the ACM (JACM), v.46 n.5, p.604-632, Sept. 1999 [doi>10.1145/324133.324140]
• 14: J.La .erty and C.Zhai.Probabilistic IR models based on query and document generation.In Proceedings of the Workshop on Language Modeling and Information Retrieval Carnegie Mellon University,May 31 -June 1,2001.
• 15: David R. H. Miller , Tim Leek , Richard M. Schwartz, A hidden Markov model information retrieval system, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.214-221, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312680]
• 16: F.Mosteller and D.Wallace.Inference and disputed authorship: The Federalist Addison Wesley,1964.
• 17: Jay M. Ponte , W. Bruce Croft, A language modeling approach to information retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.275-281, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291008]
• 18: S.Robertson and K.Sparck Jones.Relevance weighting of search terms.Journal of the American Society for Information Science 27:129 -146,1976.
• 19: S.E.Robertson,S.Walker,S.Jones,M.M.Hancock- Beaulieu,andM.Gatford.OkapiatTREC-3.InD.K.Harman,editor,The Third Text REtrieval Conference (TREC- 3) ,1995.
• 20: Ian H. Witten , Alistair Moffat , Timothy C. Bell, Managing gigabytes (2nd ed.): compressing and indexing documents and images, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1999
• 21: S. K. M. Wong , Y. Y. Yao, A probability distribution model for information retrieval, Information Processing and Management: an International Journal, v.25 n.1, p.39-53, 1989 [doi>10.1016/0306-4573(89)90090-3]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.7 DOCUMENT AND TEXT PROCESSING
      I.7.5 Document Capture
          Subjects: Document analysis

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Linguistic processing
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models; Query formulation

General Terms:
Design, Experimentation, Languages, Performance