Relevance feedback revisited

ABSTRACT

Researchers have found relevance feedback to be effective in interactive information retrieval, although few formal user experiments have been made. In order to run a user experiment on a large document collection, experiments were performed at NIST to complete some of the missing links found in using the probabilistic retrieval model. These experiments, using the Cranfield 1400 collection, showed the importance of query expansion in addition to query reweighting, and showed that adding as few as 20 well-selected terms could result in performance improvements of over 100%. Additionally it was shown that performing multiple iterations of feedback is highly effective.

REFERENCES

• 1: Bookstein A. (1983). Information Retrieval: A Sequential Learning Process. Journal of the American Society for Information Science, 34(5), 331-342.
• 2: Campbell I., Sanderson M. & van Rijsbergen K. (1990). News Retrieval Tool Technical Notes, Glasgow University: Computing Science Department.
• 3: Croft W.B. (1983). Experiments with Representation in a Document Retrieval System. information Technology: Research and Development, 2(1), 1-21.
• 4: Croft W.B. and Harper D.J. (1979). Using Probabilistic Models of Document Retrieval Without Relevance Information. Journal of Documentation, 35(4), 285-295.
• 5: Croft W.B. (1992). Personal communication.
• 6: Doszkocs T.E. (1978). A/D, an Associative Interactive Dictionary for Online Searching. Online Review, 2(2), 163-172.
• 7: D. Harman, Towards interactive query expansion, Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, p.321-331, May 1988, Grenoble, France [doi>10.1145/62437.62469]
• 8: Harman D. and Candela G. (1990). Retrieving Records from a Gigabyte of Text on a Minicomputer using Statistical Ranking. Journal of the American Society for Information Science, 41(8), 581-589.
• 9: Harper D.J. (1980). Relevance Feedback in Document Retrieval Systems: An Evaluation of ProbabUistic Strategies. Doctoral Dissertiort, Jesus College, Cambridge, England.
• 10: Harper D.J. and Van Rijsbergen C.J.(1978). An Evaluation of Feedback in Document Retrieval Using Co-Occurrence Data. Journal of Documentation, 34(3), 189-216.
• 11: Ide E. (1971). New Experiments in Relevance Feedback. In Salton G. (Ed.), The SMART Retrieval System (pp. 337-354). Englewood Cliffs, N.J.: Prentice-Hall, inc.
• 12: Lancaster F.W. (1969). MEDLARS: Report on the Evaluation of Its Operating Efficiency. American Documentation, 20(2) 119-148.
• 13: Martin Porter , Valerie Galpin, Relevance feedback in a public access catalogue for a research library: MUSCAT at the Scott Polar Research Institute Library, Program, v.22 n.1, p.1-20, January 1988 [doi>10.1108/eb046983]
• 14: Rocchio J.J. (1971). Relevance Feedback in Information Retrieval. In Salton G. (Ed.), The SMART Retrieval System (pp. 313-323). Englewood Cliffs, N.J.: Prentice-Hall, Inc.
• 15: Robertson S.E. and Sparck Jones K. (1976). Relevance Weighting of Search Terms. Journal of the American Society for Information Science, 27(3), 129-146.
• 16: S. E. Robertson, On term selection for query expansion, Journal of Documentation, v.46 n.4, p.359-364, Dec. 1990 [doi>10.1108/eb026866]
• 17: Salton G. (1970). Evaluation Problems in Interactive Information Retrieval. Information Storage and Retrieval, 6(1), 29-44.
• 18: Salton G. (1971). The SMART Retrieval System. Englewood Cliffs, N.J.: Prentice-Hail, Inc.
• 19: Salton G. and Buckley C. (1990). Improving Retrieval Performance by Relevance Feedback. Journal of the American Society for Information Science, 41 (4), 288-297.
• 20: Smeaton A.F. and van Rijsbergen C.J. (1983). The Retrieval Effects of Query Expansion on a Feedback Document Retrieval System. The Computer Journal, 26(3), 239-246. 1988.
• 21: Sparck Jones K. (1979). Search Term Relevance Weighting Given Little Relevance Information. Journal of Documentation, 35(1), 30-48.
• 22: C. J. van Rijsbergen, (invited paper) A new theoretical framework for information retrieval, Proceedings of the 9th annual international ACM SIGIR conference on Research and development in information retrieval, p.194-200, September 1986, Palazzo dei Congressi, Pisa, Italy [doi>10.1145/253168.253208]
• 23: Wu H. and Salton G. (1981). The Estimation of Term Relevance Weights using Relevance Feedback. Journal of Documentation, 37(4), 194-214.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Query formulation

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Experimentation, Human Factors, Performance