Efficient bayesian hierarchical user modeling for recommendation system

ABSTRACT

A content-based personalized recommendation system learns user specific profiles from user feedback so that it can deliver information tailored to each individual user's interest. A system serving millions of users can learn a better user profile for a new user, or a user with little feedback, by borrowing information from other users through the use of a Bayesian hierarchical model. Learning the model parameters to optimize the joint data likelihood from millions of users is very computationally expensive. The commonly used EM algorithm converges very slowly due to the sparseness of the data in IR applications. This paper proposes a new fast learning technique to learn a large number of individual user profiles. The efficacy and efficiency of the proposed algorithm are justified by theory and demonstrated on actual user data from Netflix and MovieLens.

REFERENCES

• 1: Chumki Basu , Haym Hirsh , William Cohen, Recommendation as classification: using social and content-based information in recommendation, Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence, p.714-720, July 1998, Madison, Wisconsin, United States
• 2: J. S. Breese, D. Heckerman, and C. Kadie. Empirical analysis of predictive algorithms for collaborative filtering. Technical report, Microsoft Research, One Microsoft Way, Redmond, WA 98052, 1998.
• 3: Jamie Callan, Document filtering with inference networks, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.262-269, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243273]
• 4: N. Cancedda, N. Cesa-Bianchi, A. Conconi, C. Gentile, C. Goutte, T. Graepel, Y. Li, J. M. Renders, J. S. Taylor, and A. Vinokourov. Kernel method for document filtering. In The Eleventh Text REtrieval Conference (TREC11). National Institute of Standards and Technology, special publication 500--249, 2003.
• 5: C. Chelba and A. Acero. Adaptation of maximum entropy capitalizer: Little data can help a lot. In D. Lin and D. Wu, editors, Proceedings of EMNLP 2004, pages 285--292, Barcelona, Spain, July 2004. Association for Computational Linguistics.
• 6: W. Bruce Croft , John Lafferty, Language Modeling for Information Retrieval, Kluwer Academic Publishers, Norwell, MA, 2003
• 7: Aynur Dayanik , David D. Lewis , David Madigan , Vladimir Menkov , Alexander Genkin, Constructing informative prior distributions from domain knowledge in text classification, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA [doi>10.1145/1148170.1148255]
• 8: J. Delgado and N. Ishii. Memory-based weighted majority prediction for recommender systems. In ACM SIGIR'99 Workshop on Recommender Systems, 1999.
• 9: GroupLens. Movielens. http://www.grouplens.org/taxonomy/term/14, 2006.
• 10: David Heckerman, A tutorial on learning with Bayesian networks, Learning in graphical models, MIT Press, Cambridge, MA, 1999
• 11: Jonathan L. Herlocker , Joseph A. Konstan , Al Borchers , John Riedl, An algorithmic framework for performing collaborative filtering, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.230-237, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312682]
• 12: Thomas Hofmann , Jan Puzicha, Latent Class Models for Collaborative Filtering, Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, p.688-693, July 31-August 06, 1999
• 13: I. M. D. (IMDB). Internet movie database. http://www.imdb.com/interfaces/, 2006.
• 14: Rong Jin , Joyce Y. Chai , Luo Si, An automatic weighting scheme for collaborative filtering, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009051]
• 15: Joseph A. Konstan , Bradley N. Miller , David Maltz , Jonathan L. Herlocker , Lee R. Gordon , John Riedl, GroupLens: applying collaborative filtering to Usenet news, Communications of the ACM, v.40 n.3, p.77-87, March 1997 [doi>10.1145/245108.245126]
• 16: D. Lewis. Applying support vector machines to the TREC-2001 batch filtering and routing tasks. In Proceedings of the Eleventh Text REtrieval Conference (TREC-11), 2002.
• 17: B. Liu, X. Li, W. S. Lee, , and P. Yu. Text classification by labeling words. In Proceedings of The Nineteenth National Conference on Artificial Intelligence (AAAI-2004), July 25-29, 2004.
• 18: Prem Melville , Raymod J. Mooney , Ramadass Nagarajan, Content-boosted collaborative filtering for improved recommendations, Eighteenth national conference on Artificial intelligence, p.187-192, July 28-August 01, 2002, Edmonton, Alberta, Canada
• 19: Netflix. Netflix prize. http://www.netflixprize.com (visited on Nov. 30, 2006), 2006.
• 20: S. Robertson and K. Sparck-Jones. Relevance weighting of search terms. In Journal of the American Society for Information Science, volume 27, pages 129--146, 1976.
• 21: Jun Wang , Arjen P. de Vries , Marcel J. T. Reinders, Unifying user-based and item-based collaborative filtering approaches by similarity fusion, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA [doi>10.1145/1148170.1148257]
• 22: Xiaoyun Wu , Rohini Srihari, Incorporating prior knowledge with weighted margin support vector machines, Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, 2004, Seattle, WA, USA [doi>10.1145/1014052.1014089]
• 23: Yiming Yang , Shinjae Yoo , Jian Zhang , Bryan Kisiel, Robustness of adaptive filtering methods in a cross-benchmark evaluation, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076054]
• 24: Kai Yu , Volker Tresp , Anton Schwaighofer, Learning Gaussian processes from multiple tasks, Proceedings of the 22nd international conference on Machine learning, p.1012-1019, August 07-11, 2005, Bonn, Germany [doi>10.1145/1102351.1102479]
• 25: Kai Yu , Volker Tresp , Shipeng Yu, A nonparametric hierarchical bayesian framework for information filtering, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009053]
• 26: X. Zhu. Semi-supervised learning literature survey. Technical report, University of Wisconsin - Madison, December 9, 2006.
• 27: Philip Zigoris , Yi Zhang, Bayesian adaptive user profiling with explicit & implicit feedback, Proceedings of the 15th ACM international conference on Information and knowledge management, November 06-11, 2006, Arlington, Virginia, USA [doi>10.1145/1183614.1183672]

INDEX TERMS

Primary Classification:
  B. Hardware
  B.3 MEMORY STRUCTURES
      B.3.3 Performance Analysis and Design Aids**

General Terms:
Algorithms

Keywords:
Bayesian Hierarchical models, EM algorithm, information filtering, personalization, recommender systems