Study of the usefulness of known and new implicit indicators and their optimal combination for accurate inference of users interests

ABSTRACT

Explicit relevance feedback involves explicit ratings of documents or terms by users and disrupts their browsing and searching. The alternative non-disruptive method is implicit feedback inferring users' needs and interests by monitoring their regular interaction with the system. Some implicit indicators of interest, such as reading time, have been investigated in previous studies and were found indicative to the relevance of documents but not sufficiently accurate [1,2,3,4]. In this paper we present and examine several new relative implicit feedback indicators, and study the effect of combining several implicit indicators. The paper describes a large-scale user study on which users' searches were observed by a specially developed browser that recorded their behavior (implicit indicators) as well as their explicit ratings. We analyzed the relationship between implicit indicators and explicit ratings and found that a certain combination of implicit indicators achieved higher correlation with the explicit ratings than any of the individual indicators. We have also found that the relative indicators are more indicative to the level of interest of a user item than the non-relative indicators.

REFERENCES

• 1: Mark Claypool , Phong Le , Makoto Wased , David Brown, Implicit interest indicators, Proceedings of the 6th international conference on Intelligent user interfaces, p.33-40, January 14-17, 2001, Santa Fe, New Mexico, United States [doi>10.1145/359784.359836]
• 2: Diane Kelly , Jaime Teevan, Implicit feedback for inferring user preference: a bibliography, ACM SIGIR Forum, v.37 n.2, p.18-28, Fall 2003 [doi>10.1145/959258.959260]
• 3: Oard, D. W., and Kim., J., Implicit feedback for recommender systems. In proceedings of the AAAI Workshop on Recommender Systems. 1998
• 4: Oard, D. W., and Kim, J. Modeling information content using observable behavior. In proceeding of the 64th Annual meeting of the American Society for Information Science and Technology, USA, 38--45. 2001.
• 5: Montgomery, D. C. Design and Analysis of Experiments. Wiley & Sons, Inc. (ed). 2001.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Relevance feedback

General Terms:
Experimentation, Human Factors, Measurement

Keywords:
implicit and explicit relevance feedback, user studies