Getting to know you: learning new user preferences in recommender systems

ABSTRACT

Recommender systems have become valuable resources for users seeking intelligent ways to search through the enormous volume of information available to them. One crucial unsolved problem for recommender systems is how best to learn about a new user. In this paper we study six techniques that collaborative filtering recommender systems can use to learn about new users. These techniques select a sequence of items for the collaborative filtering system to present to each new user for rating. The techniques include the use of information theory to select the items that will give the most value to the recommender system, aggregate statistics to select the items the user is most likely to have an opinion about, balanced techniques that seek to maximize the expected number of bits learned per presented item, and personalized techniques that predict which items a user will have an opinion about. We study the techniques thru offline experiments with a large pre-existing user data set, and thru a live experiment with over 300 users. We show that the choice of learning technique significantly affects the user experience, in both the user effort and the accuracy of the resulting predictions.

REFERENCES

• 1: Avery, C., Resnick, P., and Zeckhauser, R. The Market for Evaluations. American Economic Review, 89(3): 564-584.
• 2: Marko Balabanović , Yoav Shoham, Fab: content-based, collaborative recommendation, Communications of the ACM, v.40 n.3, p.66-72, March 1997 [doi>10.1145/245108.245124]
• 3: Ben-Bassat, M. Myopic Policies in Sequential Classification. IEEE Transactions on Computers, 27(2), 170-174.
• 4: Eric J. Glover , William P. Birmingham, Using decision theory to order documents, Proceedings of the third ACM conference on Digital libraries, p.285-286, June 23-26, 1998, Pittsburgh, Pennsylvania, United States [doi>10.1145/276675.276732]
• 5: Ken Goldberg , Theresa Roeder , Dhruv Gupta , Chris Perkins, Eigentaste: A Constant Time Collaborative Filtering Algorithm, Information Retrieval, v.4 n.2, p.133-151, July 2001 [doi>10.1023/A:1011419012209]
• 6: Nathaniel Good , J. Ben Schafer , Joseph A. Konstan , Al Borchers , Badrul Sarwar , Jon Herlocker , John Riedl, Combining collaborative filtering with personal agents for better recommendations, Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference innovative applications of artificial intelligence, p.439-446, July 18-22, 1999, Orlando, Florida, United States
• 7: Warren R. Greiff , Jay M. Ponte, The maximum entropy approach and probabilistic IR models, ACM Transactions on Information Systems (TOIS), v.18 n.3, p.246-287, July 2000 [doi>10.1145/352595.352597]
• 8: Ha, V., and Haddawy P. Towards Case-Based Preference Elicitation: Similarity Measures on Preference Structures. Proceedings UAI 1998, 193-201.
• 9: Horvitz, E., Heckerman D., Ng, K, Nathwani, B. Towards Normative Expert Systems: Part I, Pathfinder Project. Methods of Information in Medicine, 31, 90- 105.
• 10: Paul B. Kantor , Jung Jin Lee, The maximum entropy principle in information retrieval, Proceedings of the 9th annual international ACM SIGIR conference on Research and development in information retrieval, p.269-274, September 1986, Palazzo dei Congressi, Pisa, Italy [doi>10.1145/253168.253225]
• 11: George Karypis, Evaluation of Item-Based Top-N Recommendation Algorithms, Proceedings of the tenth international conference on Information and knowledge management, October 05-10, 2001, Atlanta, Georgia, USA [doi>10.1145/502585.502627]
• 12: Kohrs, A., and Merialdo, B. Improving Collaborative Filtering for New Users by Smart Object Selection, Proceedings of International Conference on Media Features (ICMF) 2001 (oral presentation).
• 13: Nguyen, H., and Haddawy, P. The Decision-Theoretic Video Advisor. Proceedings of AAAI Workshop on Recommender Systems, 76-80, 1998.
• 14: David M. Pennock , Eric Horvitz , Steve Lawrence , C. Lee Giles, Collaborative Filtering by Personality Diagnosis: A Hybrid Memory and Model-Based Approach, Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, p.473-480, June 30-July 03, 2000
• 15: J. Ben Schafer , Joseph A. Konstan , John Riedl, E-Commerce Recommendation Applications, Data Mining and Knowledge Discovery, v.5 n.1-2, p.115-153, January-April 2001 [doi>10.1023/A:1009804230409]
• 16: Vetschera, R. Entropy and the Value of Information, Central European Journal of Operations Research 8, 2000 S. 195-208.
• 17: Ahmad M. Ahmad Wasfi, Collecting user access patterns for building user profiles and collaborative filtering, Proceedings of the 4th international conference on Intelligent user interfaces, p.57-64, January 05-08, 1999, Los Angeles, California, United States [doi>10.1145/291080.291091]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: User interfaces

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
          Subjects: Concept learning

General Terms:
Design, Human Factors, Performance, Theory

Keywords:
collaborative filtering, entropy, information filtering, recommender systems, startup problem, user modeling