The role of variance in term weighting for probabilistic information retrieval

ABSTRACT

In probabilistic approaches to information retrieval, the occurrence of a query term in a document contributes to the probability that the document will be judged relevant. It is typically assumed that the weight assigned to a query term should be based on the expected value of that contribution. In this paper we show that the degree to which observable document features such as term frequencies are expected to vary is also important. By means of stochastic simulation, we show that increased variance results in degraded retrieval performance. We further show that by decreasing term weights in the presence of variance, this degradation can be reduced. Hence, probabilistic models of information retrieval must take into account not only the expected value of a query term's contribution but also the variance of document features.

REFERENCES

• 1: Adam Berger , John Lafferty, Information retrieval as statistical translation, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.222-229, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312681]
• 2: W. S. Cooper, A. Chen, and F. C. Gey. Full text retrieval based on probabilistic equations with coefficients fitted by logistic regression. In D. K. Harman, editor, The Second Text REtreival Conference (TREC-2), pages 57--66, Gaithersburg, Md., Mar. 1994. NIST Special Publication 500-215.
• 3: William S. Cooper , Fredric C. Gey , Daniel P. Dabney, Probabilistic retrieval based on staged logistic regression, Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, p.198-210, June 21-24, 1992, Copenhagen, Denmark [doi>10.1145/133160.133199]
• 4: W. B. Croft and D. J. Harper. Using probabilistic models of document retrieval without relevance information. Journal of Documentation, 35(4):285--295, Dec. 1979.
• 5: W. A. Gale and G. Sampson. Good-turing estimation without tears. Journal of Quantitative Linguistics, 2(3):217--237, 1995.
• 6: Fredric C. Gey, Inferring probability of relevance using the method of logistic regression, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.222-231, July 03-06, 1994, Dublin, Ireland
• 7: I. J. Good. The population of frequencies and the estimation of population parameters. Biometrica, 40:237--264, 1953.
• 8: Djoerd Hiemstra, A Linguistically Motivated Probabilistic Model of Information Retrieval, Proceedings of the Second European Conference on Research and Advanced Technology for Digital Libraries, p.569-584, September 21-23, 1998
• 9: D. Hiemstra and A. P. de~Vries. Relating the new language models of information retrieval to the traditional retrieval models. Technical Report TR-CTIT-00-09, Centre for Telematics and Information Technology, University of Twente, May 2000.
• 10: D. W. Hosmer, Jr and S. Lemeshow. Applied Logistic Regression. John Wiley & Sons, New York, 1989.
• 11: M. E. Maron , J. L. Kuhns, On Relevance, Probabilistic Indexing and Information Retrieval, Journal of the ACM (JACM), v.7 n.3, p.216-244, July 1960 [doi>10.1145/321033.321035]
• 12: David R. H. Miller , Tim Leek , Richard M. Schwartz, A hidden Markov model information retrieval system, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.214-221, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312680]
• 13: Jay M. Ponte , W. Bruce Croft, A language modeling approach to information retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.275-281, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291008]
• 14: S. E. Robertson. The probability ranking principle in IR. Journal of Documentation, 33:294--304, 1977.
• 15: S. E. Robertson and K. Sparck Jones. Relevance weighting of search terms. Journal of the American Society for Information Science, 27:129--146, 1977.
• 16: S. E. Robertson , S. Walker, Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.232-241, July 03-06, 1994, Dublin, Ireland
• 17: S. E. Robertson , S. Walker , M. M. Hancock-Beaulieu, Large test collection experiments on an operational, interactive system: Okapi at TREC, Information Processing and Management: an International Journal, v.31 n.3, p.345-360, May-June 1995
• 18: Fei Song , W. Bruce Croft, A general language model for information retrieval, Proceedings of the eighth international conference on Information and knowledge management, p.316-321, November 02-06, 1999, Kansas City, Missouri, United States [doi>10.1145/319950.320022]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING
      I.6.3 Applications

General Terms:
Performance, Theory

Keywords:
latent variables, log-linear models, parameter estimation, term weighting, variance