A study of factors affecting the utility of implicit relevance feedback

ABSTRACT

Implicit relevance feedback (IRF) is the process by which a search system unobtrusively gathers evidence on searcher interests from their interaction with the system. IRF is a new method of gathering information on user interest and, if IRF is to be used in operational IR systems, it is important to establish when it performs well and when it performs poorly. In this paper we investigate how the use and effectiveness of IRF is affected by three factors: search task complexity, the search experience of the user and the stage in the search. Our findings suggest that all three of these factors contribute to the utility of IRF.

REFERENCES

• 1: Bell, D.J. and Ruthven, I. (2004). Searchers' assessments of task complexity for web searching. Proceedings of the 26th European Conference on Information Retrieval, 57--71.
• 2: Borlund, P. (2000). Experimental components for the evaluation of interactive information retrieval systems. Journal of Documentation. 56(1): 71--90.
• 3: Giorgio Brajnik , Stefano Mizzaro , Carlo Tasso, Strategic help in user interfaces for information retrieval, Journal of the American Society for Information Science and Technology, v.53 n.5, p.343-358, May 2002 [doi>10.1002/asi.10035]
• 4: Busha, C.H. and Harter, S.P., (1980). Research methods in librarianship: Techniques and interpretation. Library and information science series. New York: Academic Press.
• 5: Campbell, I. and Van Rijsbergen, C.J. (1996). The ostensive model of developing information needs. Proceedings of the 3rd International Conference on Conceptions of Library and Information Science, 251--268.
• 6: Donna Harman, Relevance feedback and other query modification techniques, Information retrieval: data structures and algorithms, Prentice-Hall, Inc., Upper Saddle River, NJ, 1992
• 7: Diane Kelly , Jaime Teevan, Implicit feedback for inferring user preference: a bibliography, ACM SIGIR Forum, v.37 n.2, p.18-28, Fall 2003 [doi>10.1145/959258.959260]
• 8: Jürgen Koenemann , Nicholas J. Belkin, A case for interaction: a study of interactive information retrieval behavior and effectiveness, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.205-212, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238487]
• 9: Meddis, R., (1984). Statistics using ranks: A unified approach. Oxford: Basil Blackwell, 303--308.
• 10: Masahiro Morita , Yoichi Shinoda, Information filtering based on user behavior analysis and best match text retrieval, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.272-281, July 03-06, 1994, Dublin, Ireland
• 11: Salton, G. and Buckley, C. (1990). Improving retrieval performance by relevance feedback. Journal of the American Society for Information Science. 41(4): 288--297.
• 12: Siegel, S. and Castellan, N.J. (1988). Nonparametric statistics for the behavioural sciences. 2nd ed. Singapore: McGraw-Hill.
• 13: White, R.W. (2004). Implicit feedback for interactive information retrieval. Unpublished Doctoral Dissertation, University of Glasgow, Glasgow, United Kingdom.
• 14: Ryen W. White, Implicit feedback for interactive information retrieval, ACM SIGIR Forum, v.39 n.1, June 2005 [doi>10.1145/1067268.1067297]
• 15: White, R.W., Jose, J.M., Ruthven, I. and Van Rijsbergen, C.J. (2004). A simulated study of implicit feedback models. Proceedings of the 26th European Conference on Information Retrieval, 311--326.
• 16: Polle T. Zellweger , Susan Harkness Regli , Jock D. Mackinlay , Bay-Wei Chang, The impact of fluid documents on reading and browsing: an observational study, Proceedings of the SIGCHI conference on Human factors in computing systems, p.249-256, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332440]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Experimentation, Human Factors

Keywords:
implicit relevance feedback, relevance feedback