Relevance based language models

ABSTRACT

We explore the relation between classical probabilistic models of information retrieval and the emerging language modeling approaches. It has long been recognized that the primary obstacle to effective performance of classical models is the need to estimate arelevance model: probabilities of words in the relevant class. We propose a novel technique for estimating these probabilities using the query alone. We demonstrate that our technique can produce highly accurate relevance models, addressing important notions of synonymy and polysemy. Our experiments show relevance models outperforming baseline language modeling systems on TREC retrieval and TDT tracking tasks. The main contribution of this work is an effective formal method for estimating a relevance model with no training data.

REFERENCES

• 1: J. Allan, J. Callan, F. Feng, and D. Malin. INQUERY and TREC-8. In D. Harman, editor, Proceedings of the Eighth Text REtrieval Conference (TREC-8), 1999.
• 2: James Allan , Ron Papka , Victor Lavrenko, On-line new event detection and tracking, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.37-45, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290954]
• 3: Doug Beeferman , Adam Berger , John Lafferty, Statistical Models for Text Segmentation, Machine Learning, v.34 n.1-3, p.177-210, Feb. 1999 [doi>10.1023/A:1007506220214]
• 4: Adam Berger , Rich Caruana , David Cohn , Dayne Freitag , Vibhu Mittal, Bridging the lexical chasm: statistical approaches to answer-finding, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.192-199, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345576]
• 5: Adam Berger , John Lafferty, Information retrieval as statistical translation, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.222-229, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312681]
• 6: Adam L. Berger , Vibhu O. Mittal, OCELOT: a system for summarizing Web pages, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.144-151, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345565]
• 7: Peter F. Brown , Vincent J. Della Pietra , Stephen A. Della Pietra , Robert L. Mercer, The mathematics of statistical machine translation: parameter estimation, Computational Linguistics, v.19 n.2, June 1993
• 8: Stanley F. Chen , Joshua Goodman, An empirical study of smoothing techniques for language modeling, Proceedings of the 34th annual meeting on Association for Computational Linguistics, p.310-318, June 24-27, 1996, Santa Cruz, California [doi>10.3115/981863.981904]
• 9: C. Cieri, D.Graff, M.Liberman, N.Martey, and S.Strassel. The TDT-2 text and speech corpus. In Proceedings of the DARPA Broadcast News Workshop, pp 57-60, 1999.
• 10: D. Hiemstra. Using language models for information retrieval. In PhD Thesis, University of Twente, 2001.
• 11: D. Hiemstra and A. de Vries. Relating the new language models of information retrieval to the traditional retrieval models. In CTIT Technical Report TR-CTIT-00-09, 2000.
• 12: H. Jin, R. Schwartz, S. Sista, and F. Walls. Topic tracking for radio, TV broadcast and newswire. In Proceedings of DARPA Broadcast News Workshop, pp 199-204, 1999.
• 13: A. Martin, G. Doddington, T. Kamm, and M. Ordowski. The DET curve in assessment of detection task performance. In EuroSpeech, pages 1895-1898, 1997.
• 14: David R. H. Miller , Tim Leek , Richard M. Schwartz, A hidden Markov model information retrieval system, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.214-221, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312680]
• 15: Jay Michael Ponte, A language modeling approach to information retrieval, University of Massachusetts, Amherst, MA, 1998
• 16: Jay M. Ponte , W. Bruce Croft, A language modeling approach to information retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.275-281, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291008]
• 17: S. Robertson and K. S. Jones. Relevance weighting of search terms. In Journal of the American Society for Information Science, vol.27, 1977.
• 18: S. E. Robertson , S. Walker, Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.232-241, July 03-06, 1994, Dublin, Ireland
• 19: S. E. Robertson, The probability ranking principle in IR, Readings in information retrieval, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1997
• 20: S. E. Robertson, S. Walker, S. Jones, M. M. Hancock-Beaulieu, and M. Gatford. OKAPI at TREC-3. In D. Harman, editor, Proceedings of the 3rd Text REtrieval Conference (TREC-3), 1996.
• 21: Fei Song , W. Bruce Croft, A general language model for information retrieval (poster abstract), Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.279-280, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312698]
• 22: H. Turtle and W. B. Croft. Efficient probabilistic inference for text retrieval. In Proceedings of RIAO 3, pages 644-651, 1991.
• 23: C. J. van Rijsbergen. A theoretical basis for the use of co-occurrence data in information retrieval. Journal of Documentation, 33:106-119, 1977.
• 24: Jinxi Xu , W. Bruce Croft, Improving the effectiveness of information retrieval with local context analysis, ACM Transactions on Information Systems (TOIS), v.18 n.1, p.79-112, Jan. 2000 [doi>10.1145/333135.333138]
• 25: J. Yamron, I. Carp, L. Gillick, S.Lowe, and P. van Mulbregt. Topic tracking in a news stream. In Proceedings of DARPA Broadcast News Workshop, pp 133-136, 1999.
• 26: J. Yamron, S. Knecht, and P. van Mulbregt. Dragon's tracking and detection systems for the TDT2000 evaluation. In Proceedings of Topic Detection and Tracking Workshop, pp 75-80, 2000.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Relevance feedback

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Linguistic processing
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Design, Experimentation, Performance