Regression-based latent factor models

ABSTRACT

We propose a novel latent factor model to accurately predict response for large scale dyadic data in the presence of features. Our approach is based on a model that predicts response as a multiplicative function of row and column latent factors that are estimated through separate regressions on known row and column features. In fact, our model provides a single unified framework to address both cold and warm start scenarios that are commonplace in practical applications like recommender systems, online advertising, web search, etc. We provide scalable and accurate model fitting methods based on Iterated Conditional Mode and Monte Carlo EM algorithms. We show our model induces a stochastic process on the dyadic space with kernel (covariance) given by a polynomial function of features. Methods that generalize our procedure to estimate factors in an online fashion for dynamic applications are also considered. Our method is illustrated on benchmark datasets and a novel content recommendation application that arises in the context of Yahoo! Front Page. We report significant improvements over several commonly used methods on all datasets.

REFERENCES

• 1: KDD cup and workshop. 2007.
• 2: Deepak Agarwal , Bee-Chung Chen , Pradheep Elango, Spatio-temporal models for estimating click-through rate, Proceedings of the 18th international conference on World wide web, April 20-24, 2009, Madrid, Spain [doi>10.1145/1526709.1526713]
• 3: D. Agarwal and B.-C. Chen, et al. Online models for content optimization. In NIPS, 2008.
• 4: Deepak Agarwal , Srujana Merugu, Predictive discrete latent factor models for large scale dyadic data, Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, August 12-15, 2007, San Jose, California, USA [doi>10.1145/1281192.1281199]
• 5: G. Allenby, P. Rossi, and R. McCulloch. Hierarchical bayes models: A practitioner's guide. http://ssrn.com/abstract=655541, 2005.
• 6: Marko Balabanović , Yoav Shoham, Fab: content-based, collaborative recommendation, Communications of the ACM, v.40 n.3, p.66-72, March 1997 [doi>10.1145/245108.245124]
• 7: Arindam Banerjee , Inderjit Dhillon , Joydeep Ghosh , Srujana Merugu , Dharmendra S. Modha, A Generalized Maximum Entropy Approach to Bregman Co-clustering and Matrix Approximation, The Journal of Machine Learning Research, 8, p.1919-1986, 12/1/2007
• 8: Robert Bell , Yehuda Koren , Chris Volinsky, Modeling relationships at multiple scales to improve accuracy of large recommender systems, Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, August 12-15, 2007, San Jose, California, USA [doi>10.1145/1281192.1281206]
• 9: J. Booth and J. Hobert. Maximizing generalized linear mixed model likelihoods with an automated monte carlo EM algorithm. J.R.Statist. Soc. B, 1999.
• 10: M. Claypool and A. Gokhale, et al. Combining content-based and collaborative filters in an online newspaper. In Recommender Systems Workshop, 1999.
• 11: A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. J. of the Royal Statistical Society, Series B, 1977.
• 12: A. Gelman and J. Hill. Data Analysis using Regression and Multilevel/Hierarchical Models. Cambridge, 2006.
• 13: A. Gelman and A. Jakulin, et al. A weakly informative default prior distribution for logistic and other regression models. Annals of Applied Statistics, 2008.
• 14: Nathaniel Good , J. Ben Schafer , Joseph A. Konstan , Al Borchers , Badrul Sarwar , Jon Herlocker , John Riedl, Combining collaborative filtering with personal agents for better recommendations, Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference innovative applications of artificial intelligence, p.439-446, July 18-22, 1999, Orlando, Florida, United States
• 15: Jonathan L. Herlocker , Joseph A. Konstan , Al Borchers , John Riedl, An algorithmic framework for performing collaborative filtering, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.230-237, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312682]
• 16: Thomas Hofmann, Probabilistic latent semantic indexing, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.50-57, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312649]
• 17: D. L. Lee and S. Seung. Algorithms for non-negative matrix factorization. In NIPS, 2001.
• 18: P. McCullagh and J. A. Nelder. Generalized Linear Models. Chapman&Hall/CRC, 1989.
• 19: Radford M. Neal , Geoffrey E. Hinton, A view of the EM algorithm that justifies incremental, sparse, and other variants, Proceedings of the NATO Advanced Study Institute on Learning in graphical models, p.355-368, March 1998, Erice, Italy
• 20: Seung-Taek Park , David Pennock , Omid Madani , Nathan Good , Dennis DeCoste, Naïve filterbots for robust cold-start recommendations, Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, August 20-23, 2006, Philadelphia, PA, USA [doi>10.1145/1150402.1150490]
• 21: Carl Edward Rasmussen , Christopher K. I. Williams, Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning), The MIT Press, 2005
• 22: Jasson D. M. Rennie , Nathan Srebro, Fast maximum margin matrix factorization for collaborative prediction, Proceedings of the 22nd international conference on Machine learning, p.713-719, August 07-11, 2005, Bonn, Germany [doi>10.1145/1102351.1102441]
• 23: Ruslan Salakhutdinov , Andriy Mnih, Bayesian probabilistic matrix factorization using Markov chain Monte Carlo, Proceedings of the 25th international conference on Machine learning, p.880-887, July 05-09, 2008, Helsinki, Finland [doi>10.1145/1390156.1390267]
• 24: R. Salakhutdinov and A. Mnih. Probabilistic matrix factorization. In NIPS, 2008.
• 25: Andrew I. Schein , Alexandrin Popescul , Lyle H. Ungar , David M. Pennock, Methods and metrics for cold-start recommendations, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564421]
• 26: A. I. Schein, L. K. Saul, and L. H. Ungar. A generalized linear model for principal component analysis of binary data. In AISTATS, 2003.
• 27: R. Smith. Bayesian and Frequentist Approaches to Parametric Predictive Inference. Oxford University, 1999.
• 28: Yi Zhang , Jonathan Koren, Efficient bayesian hierarchical user modeling for recommendation system, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277752]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES

Additional Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS

General Terms:
Algorithms, Experimentation, Theory

Keywords:
dyadic data, interaction, latent factor, predictive, recommender systems, sparse