Matrix factorization and neighbor based algorithms for the netflix prize problem

ABSTRACT

Collaborative filtering (CF) approaches proved to be effective for recommender systems in predicting user preferences in item selection using known user ratings of items. This subfield of machine learning has gained a lot of popularity with the Netflix Prize competition started in October 2006. Two major approaches for this problem are matrix factorization (MF) and the neighbor based approach (NB). In this work, we propose various variants of MF and NB that can boost the performance of the usual ensemble based scheme. First, we investigate various regularization scenarios for MF. Second, we introduce two NB methods: one is based on correlation coefficients and the other on linear least squares. At the experimentation part, we show that the proposed approaches compare favorably with existing ones in terms of prediction accuracy and/or required training time. We present results of blending the proposed methods.

REFERENCES

• 1: Robert M. Bell , Yehuda Koren, Scalable Collaborative Filtering with Jointly Derived Neighborhood Interpolation Weights, Proceedings of the 2007 Seventh IEEE International Conference on Data Mining, p.43-52, October 28-31, 2007 [doi>10.1109/ICDM.2007.90]
• 2: R. M. Bell, Y. Koren, and C. Volinsky. The BellKor solution to the Net Flix Prize. Technical report, AT&T Labs Research, 2007. http://www.netflixprize.com/assets/ProgressPrize2007_KorBell.pdf.
• 3: James Bennett , Charles Elkan , Bing Liu , Padhraic Smyth , Domonkos Tikk, KDD Cup and workshop 2007, ACM SIGKDD Explorations Newsletter, v.9 n.2, December 2007 [doi>10.1145/1345448.1345459]
• 4: J. Bennett and S. Lanning. The Net Flix Prize. In Proc. of KDD Cup Workshop at SIGKDD'07, 13th ACM Int. Conf. on Knowledge Discovery and Data Mining, pages 3--6, San Jose, CA, USA, 2007.
• 5: J. S. Breese, D. Heckerman, and C. Kadie. Empirical analysis of predictive algorithms for collaborative filtering. In Proc. of UAI'98, 14th Conference on Uncertainty in Artificial Intelligence, pages 43--52. Morgan-Kaufmann, 1998.
• 6: John Canny, Collaborative filtering with privacy via factor analysis, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564419]
• 7: David Goldberg , David Nichols , Brian M. Oki , Douglas Terry, Using collaborative filtering to weave an information tapestry, Communications of the ACM, v.35 n.12, p.61-70, Dec. 1992 [doi>10.1145/138859.138867]
• 8: Thomas Hofmann, Latent semantic models for collaborative filtering, ACM Transactions on Information Systems (TOIS), v.22 n.1, p.89-115, January 2004 [doi>10.1145/963770.963774]
• 9: A. Paterek. Improving regularized singular value decomposition for collaborative filtering. In Proc. of KDD Cup Workshop at SIGKDD'07, 13th ACM Int. Conf. on Knowledge Discovery and Data Mining, pages 39--42, San Jose, CA, USA, 2007.
• 10: David M. Pennock , Eric Horvitz , Steve Lawrence , C. Lee Giles, Collaborative Filtering by Personality Diagnosis: A Hybrid Memory and Model-Based Approach, Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, p.473-480, June 30-July 03, 2000
• 11: A. M. Rashid, S. K. Lam, G. Karypis, and J. Riedl. ClustKNN: a highly scalable hybrid model-& memory-based CF algorithm. In Proc. of WebKDD'06: KDD Workshop on Web Mining and Web Usage Analysis, at 12th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, Philadelphia, PA, USA, 2006.
• 12: Paul Resnick , Neophytos Iacovou , Mitesh Suchak , Peter Bergstrom , John Riedl, GroupLens: an open architecture for collaborative filtering of netnews, Proceedings of the 1994 ACM conference on Computer supported cooperative work, p.175-186, October 22-26, 1994, Chapel Hill, North Carolina, United States [doi>10.1145/192844.192905]
• 13: R. Salakhutdinov and A. Mnih. Probabilistic matrix factorization. In J. C. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20, Cambridge, MA, 2008. MIT Press.
• 14: Ruslan Salakhutdinov , Andriy Mnih , Geoffrey Hinton, Restricted Boltzmann machines for collaborative filtering, Proceedings of the 24th international conference on Machine learning, p.791-798, June 20-24, 2007, Corvalis, Oregon [doi>10.1145/1273496.1273596]
• 15: B. M. Sarwar, G. Karypis, J. A. Konstan, and J. Riedl. Application of dimensionality reduction in recommender system - a case study. In Proc. of WebKDD'00: Web Mining for E-Commerce Workshop, at 6th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, Boston, MA, USA, 2000.
• 16: Badrul Sarwar , George Karypis , Joseph Konstan , John Reidl, Item-based collaborative filtering recommendation algorithms, Proceedings of the 10th international conference on World Wide Web, p.285-295, May 01-05, 2001, Hong Kong, Hong Kong [doi>10.1145/371920.372071]
• 17: N. Srebro, J. D. M. Rennie, and T. S. Jaakkola. Maximum-margin matrix factorization. Advances in Neural Information Processing Systems, 17, 2005.
• 18: G. Takács, I. Pilászy, B. Németh, and D. Tikk. On the Gravity recommendation system. In Proc. of KDD Cup Workshop at SIGKDD'07, 13th ACM Int. Conf. on Knowledge Discovery and Data Mining, pages 22--30, San Jose, CA, USA, 2007.
• 19: M. G. Vozalis , K. G. Margaritis, Using SVD and demographic data for the enhancement of generalized Collaborative Filtering, Information Sciences: an International Journal, v.177 n.15, p.3017-3037, August, 2007 [doi>10.1016/j.ins.2007.02.036]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
          Subjects: Parameter learning

General Terms:
Algorithms, Experimentation

Keywords:
collaborative filtering, matrix factorization, neighbor based methods, netflix prize