A reinforcement learning agent for personalized information filtering

ABSTRACT

This paper describes a method for learning user's interests in the Web-based personalized information filtering system called WAIR. The proposed method analyzes user's reactions to the presented documents and learns from them the profiles for the individual users. Reinforcement learning is used to adapt the term weights in the user profile so that user's preferences are best represented. In contrast to conventional relevance feedback methods which require explicit user feedbacks, our approach learns user preferences implicitly from direct observations of user behaviors during interaction. Field tests have been made which involved 7 users reading a total of 7,700 HTML documents during 4 weeks. The proposed method showed superior performance in personalized information filtering compared to the existing relevance feedback methods.

REFERENCES

• 1: Richard S. Sutton , Andrew G. Barto, Introduction to Reinforcement Learning, MIT Press, Cambridge, MA, 1998
• 2: Thomas M. Mitchell, Machine Learning, McGraw-Hill Higher Education, 1997
• 3: Gerard Salton, Automatic Tat Processing, Addison Wesley, 1989.
• 4: Nicholas J. Belkin , W. Bruce Croft, Information filtering and information retrieval: two sides of the same coin?, Communications of the ACM, v.35 n.12, p.29-38, Dec. 1992 [doi>10.1145/138859.138861]
• 5: Pattie Maes, Agents that reduce work and information overload, Communications of the ACM, v.37 n.7, p.30-40, July 1994 [doi>10.1145/176789.176792]
• 6: Henry Lieberman, Letizia: An agent that assists Web browsing, In IJCAI '95, pp. 475-480, 1995.
• 7: T. Joachims, Dayne Freitag, and ,Tom Mitchell, WebWatcher: A tour guide for the World Wide Web, In ZJCAI '97, pp. 770-777, 1997.
• 8: J.J. Rocchio, ReIevance feedback in information retrieval, In The SMART Retrieval System, Prentice Hall, pp. 313- 323,197l.
• 9: E. IDE, New experiments in relevance feedback, In The SMART Retrieval System, Prentice Hall, pp. 337-354, 1971.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE

General Terms:
Design, Experimentation, Theory

Keywords:
Web-based information filtering, learning user's preferences, reinforcement learning, user interface agents