A study of smoothing methods for language models applied to Ad Hoc information retrieval

ABSTRACT

Language modeling approaches to information retrieval are attractive and promising because they connect the problem of retrieval with that of language model estimation, which has been studied extensively in other application areas such as speech recognition. The basic idea of these approaches is to estimate a language model for each document, and then rank documents by the likelihood of the query according to the estimated language model. A core problem in language model estimation is smoothing, which adjusts the maximum likelihood estimator so as to correct the inaccuracy due to data sparseness. In this paper, we study the problem of language model smoothing and its influence on retrieval performance. We examine the sensitivity of retrieval performance to the smoothing parameters and compare several popular smoothing methods on different test collections.

REFERENCES

• 1: Adam Berger , John Lafferty, Information retrieval as statistical translation, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.222-229, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312681]
• 2: S.F.Chen and J.Goo man (1998)."An empirical study of smoothing techniques for language modeling,"Tech.Rep. TR-10-98,Harvar University.
• 3: Norbert Fuhr, Probabilistic models in information retrieval, The Computer Journal, v.35 n.3, p.243-255, June 1992 [doi>10.1093/comjnl/35.3.243]
• 4: I.J.Goo (1953)."The Population Frequencies of Species and the Estimation of Population Parameters,"Biometrika Volume 40,parts 3,4,pp.237 -264.
• 5: D.Hiemstra and W.Kraaij (1998)."Twenty-one at TREC- 7:A -hoc and cross-language track,"in Proc. of Seventh Text REtrieval Conference (TREC-7),Gaithersburg,MD.
• 6: F.Jelinek and R.Mercer (1980)."Interpolated estimation of Markov source parameters from sparse ata ".In Pattern Recognition in Practice E.S.Gelsemaan L.N.Kanal(editors),pages 381 -402.North Holland,Amsterdam.
• 7: S.M.Katz (1987)."Estimation of probabilities from sparse data for the language model component of a speech recognizer,"IEEE Transactions on Acoustics, Speech and Signal Processing volume ASSP-35,pages 400 -401,March 1987.
• 8: R.Kneser and H.Ney (1995)."Improved smoothing for mgram language modeling,"in Proceedings of the International Conference on Acoustics, Speech and Signal Processing Detroit,MI.
• 9: MacKay,D.and Peto,L.(1995)."A hierarchical Dirichlet language model."Natural Language Engineering 1(3),pp. 289 -307.
• 10: David R. H. Miller , Tim Leek , Richard M. Schwartz, A hidden Markov model information retrieval system, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.214-221, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312680]
• 11: H.Ney,U.Essen,and R.Kneser (1994)."On structuring probabilistic epen encies in stochastic language mo eling," Computer Speech and Language 8:1-38.
• 12: Jay Michael Ponte, A language modeling approach to information retrieval, University of Massachusetts, Amherst, MA, 1998
• 13: Jay M. Ponte , W. Bruce Croft, A language modeling approach to information retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.275-281, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291008]
• 14: C.J.van Rijsbergen (1986)."A Non-classical Logic for Information Retrieval,"The Computer Journal 29(6).
• 15: S. E. Robertson , C. J. van Rijsbergen , M. F. Porter, Probabilistic models of indexing and searching, Proceedings of the 3rd annual ACM conference on Research and development in information retrieval, p.35-56, June 23-27, 1980, Cambridge, England
• 16: S.E.Robertson,S.Walker,S.Jones,M.M.Hancock- Beaulieu,and M.Gatfor (1995)."Okapi at TREC-3,"The Third Text REtrieval Conference (TREC-3),inD.K.Harman (e ),NIST Special Publication.
• 17: Gerard Salton , Christopher Buckley, Term-weighting approaches in automatic text retrieval, Information Processing and Management: an International Journal, v.24 n.5, p.513-523, 1988 [doi>10.1016/0306-4573(88)90021-0]
• 18: G.Salton and C.Buckley (1990),"Improving retrieval performance by relevance feedback ",Journal of the American Society for Information Science,Vol.44,No.4,288 -297.
• 19: Amit Singhal , Chris Buckley , Mandar Mitra, Pivoted document length normalization, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.21-29, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243206]
• 20: Fei Song , W. Bruce Croft, A general language model for information retrieval (poster abstract), Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.279-280, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312698]
• 21: Karen Sparck Jones , Peter Willett, Readings in information retrieval, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1997
• 22: S. K. M. Wong , Y. Y. Yao, On modeling information retrieval with probabilistic inference, ACM Transactions on Information Systems (TOIS), v.13 n.1, p.38-68, Jan. 1995 [doi>10.1145/195705.195713]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Speech recognition and synthesis

General Terms:
Algorithms, Design, Experimentation, Performance