Bayesian extension to the language model for ad hoc information retrieval

ABSTRACT

We propose a Bayesian extension to the ad-hoc Language Model. Many smoothed estimators used for the multinomial query model in ad-hoc Language Models (including Laplace and Bayes-smoothing) are approximations to the Bayesian predictive distribution. In this paper we derive the full predictive distribution in a form amenable to implementation by classical IR models, and then compare it to other currently used estimators. In our experiments the proposed model outperforms Bayes-smoothing, and its combination with linear interpolation smoothing outperforms all other estimators.

REFERENCES

• 1: Adam Berger , John Lafferty, Information retrieval as statistical translation, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.222-229, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312681]
• 2: Stanley. F. Chen and Joshua Goodman. An empirical study of smoothing techniques for language modeling. Technical Report TR-10-98, Center for Research in Computing Technology. Harvard University, August 1998.
• 3: W. B Croft, D. J Harper, D. H Kraft, and J. Zobel, editors. SIGIR 2001: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM Press, 2001.
• 4: A. Gelman, J. B. Carlin, H. S. Stern, and D. B. Rubin. Bayesian Data Analysis. Chapman and Hall/CRC, 1885.
• 5: D. Hiemstra and W. Kraaij. Twenty-one at trec-7: ad-hoc and cross-language track. In Voorhees and Harman {11}, pages 227--238. NIST Special Publication 500--242.
• 6: Victor Lavrenko , W. Bruce Croft, Relevance based language models, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.120-127, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383972]
• 7: David J. C. McKay and Linda C. Bauman Peto. A hierarchical dirichlet language model. Natural Language Engineering, 1(3):289--307, 1995.
• 8: D. Miller, T. Leek, and R. Schwartz. Bbn at trec-7: using hidden markov models for information retrieval. In Voorhees and Harman {11}, pages 133--142. NIST Special Publication 500--242.
• 9: Jay M. Ponte , W. Bruce Croft, A language modeling approach to information retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.275-281, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291008]
• 10: S. E. Robertson and D. Hiemstra. Language models and probability of relevance'. In Proceedings of the first Workshop on Language Modeling and Information Retrieval, pages 21--25, 2001.
• 11: E. M. Voorhees and D. K. Harman, editors. The Seventh Text REtrieval Conference (TREC--7). Gaithersburg, MD: NIST, 1999. NIST Special Publication 500--242.
• 12: Grace Wahba. Spline Models for Observational Data, volume. 59. SIAM, 1992.
• 13: Chengxiang Zhai , John Lafferty, A study of smoothing methods for language models applied to Ad Hoc information retrieval, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.334-342, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.384019]
• 14: ChengXiang Zhai , John Lafferty, Two-stage language models for information retrieval, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564387]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Algorithms, Performance, Theory

Keywords:
Bayesian language model, ad hoc language model, ad hoc retrieval, information retrieval