An experimental study of factors important in document ranking

ABSTRACT

The ability to effectively rank retrieved documents in order of their probable relevance to a query is a critical factor in statistically-based keyword retrieval systems. This paper summarizes a set of experiments with different methods of term weighting for documents, using measures of term importance within an entire document collection, term importance within a given document, and document length. It is shown that significant improvements over no term weighting can be made using a combination of weighting measures and normalizing for document length.

REFERENCES

• CROFT79: Croft W.B., Harper D.I., "Using Probabilistic Models of Document R~tricval Without Relevance Information", Journal of Documentation, Vol. 35, No. 4, #mbe.r 1979, pp. 285-295.
• CROFT82: Croft W.B., "Experiments with Roprcscntation in a Document Retrieval System", COIN Technical Report 82-21, May 1982.
• DENNIS64: Dennis S.F., "The Construction of a Thesaurus Automat/cally from a Sample of Tcxt",Symposium Proc#gs, Statistical Association Methods for Mecb# Documentation, 1964. (National Bureau of Standards Miscellaneous Publication 269).
• HARTER75: Haner $.P., "A Probabilisti~ Approach to Automatic Keyword Indexing", Journal of the Amesican Society for Information Science, VoL 26, No. 5, Octobe# 1973, pp. 280-289.
• MCGILL79: McGiU M., Koll M., Noreault T., "An Evaluation of Factors Affecting Document Ranking by Information Retrieval Systems". Report, School of Inforration Smd/es, Syracuse University, Syracuse, New York, October 1979.
• RAGHAVEN83: Vijay V. Raghavan , Hong-pao Shi , C. T. Yu, Evaluation of the 2-Poisson model as a basis for using term frequency data in searching, Proceedings of the 6th annual international ACM SIGIR conference on Research and development in information retrieval, June 06-08, 1983, Bethesda, Maryland
• SALTON73: Salton G., Yang C.S., "On the Specilica#on of Term Values in Automti~ hdcxing", Journal of Documentation, Vol. 29, No. 4, December 1973, pp. 351-372.
• SALTON83: Gerard Salton , Michael J. McGill, Introduction to Modern Information Retrieval, McGraw-Hill, Inc., New York, NY, 1986
• SPARCK72: Sparck Jones K., "A Statistical Intmtn#tation of Term Specificity and Its Application in Retrieval", Your. hal of D ocungntation, Vol. 28, No. 1, March 1972, pp. 11- 20.
• YU82: C. T. Yu , K. Lam , G. Salton, Term Weighting in Information Retrieval Using the Term Precision Model, Journal of the ACM (JACM), v.29 n.1, p.152-170, Jan. 1982 [doi>10.1145/322290.322300]