A spatio-temporal approach to collaborative filtering

ABSTRACT

In this paper, we propose a novel spatio-temporal model for collaborative filtering applications. Our model is based on low-rank matrix factorization that uses a spatio-temporal filtering approach to estimate user and item factors. The spatial component regularizes the factors by exploiting correlation across users and/or items, modeled as a function of some implicit feedback (e.g., who rated what) and/or some side information (e.g., user demographics, browsing history). In particular, we incorporate correlation in factors through a Markov random field prior in a probabilistic framework, whereby the neighborhood weights are functions of user and item covariates. The temporal component ensures that the user/item factors adapt to process changes that occur through time and is implemented in a state space framework with fast estimation through Kalman filtering. Our spatio-temporal filtering (ST-KF hereafter) approach provides a single joint model to simultaneously incorporate both spatial and temporal structure in ratings and therefore provides an accurate method to predict future ratings. To ensure scalability of ST-KF, we employ a mean-field approximation for inference. Incorporating user/item covariates in estimating neighborhood weights also helps in dealing with both cold-start and warm-start problems seamlessly in a single unified modeling framework; covariates predict factors for new users and items through the neighborhood. We illustrate our method on simulated data, benchmark data and data obtained from a relatively new recommender system application arising in the context of Yahoo! Front Page.

REFERENCES

• 1: J. Abernethy, F. Bach, T. Evgeniou, and J.-P. Vert. A new approach to collaborative filtering: Operator estimation with spectral regularization. JMLR, 2009.
• 2: D. Agarwal and B.-C. Chen. Regression-based latent factor models. In KDD, 2009.
• 3: D. Agarwal, B.-C. Chen, and P. Elango. Spatio-temporal models for estimating click-rate. In WWW, 2009.
• 4: D. Agarwal, B.-C. Chen, P. Elango, R. Ramakrishnan, N. Motgi, S. Roy, and J. Zachariah. Online models for content optimization. In NIPS(21), 2009.
• 5: D. Agarwal and S. Merugu. Predictive discrete latent factor models for large scale dyadic data. In KDD, 2007.
• 6: J. Basilico and T. Hofmann. Unifying collaborative and content-based filtering. In ICML, 2004.
• 7: D. Chakrabarti, D. Agarwal, and V. Josifovski. Contextual advertising by combining relevance with click feedback. In WWW, 2008.
• 8: C. Chui and G. Chen. Kalman Filtering for Real Time Application. Springer-Verlag, 1999.
• 9: N. Cristianini, J. Kandola, A. Elisseeff, and J. Shawe-Taylor. On kernel-target. In NIPS(14), 2002.
• 10: D.Stern, R.Herbrich, and G.Thore. Matchbox: Large scale online Bayesian recommendations. In WWW, 2009.
• 11: T. S. Jaakkola. Tutorial on variational approximation methods. In Advanced Mean Field Methods: Theory and Practice, pages 129--159. MIT Press, 2000.
• 12: J.Besag. Spatial interaction and the statistical analysis of lattice systems. J. Roy. Stat. Soc. B, 36(2):192--236, 1974.
• 13: D. D. Lee and H. S. Seung. Algorithms for non-negative matrix factorization. In NIPS(12), 2000.
• 14: W. Li and D. Y. Relation regularized matrix factorization. In IJCAI, 2009.
• 15: Z. Lu, M. A. Carreira-Perpinan, and C. Sminchisescu. People tracking with the Laplacian eigenmaps latent variable model. In NIPS(20), 2008.
• 16: B. M. Marlin, R. S. Zemel, S. Roweis, and M. Slaney. Collaborative filtering and the missing at random assumption. In UAI, 2007.
• 17: R. Salakhutdinov and A. Mnih. Probabilistic matrix factorization. In NIPS(20), 2008.
• 18: N. Srebro and T. Jaakkola. Weighted low-rank approximations. In ICML, 2003.
• 19: N. Srebro, J. Rennie, and T. Jaakkola. Maximum margin matrix factorization. In NIPS(17), 2005.
• 20: R. van der Merwe. Sigma-Point Kalman Filters for Probabilistic Inference in Dynamic State-Space Models. PhD thesis, OGI, OHSU, 2004.
• 21: K. Yu and W. Chu. Gaussian process models for link analysis and transfer learning. In NIPS(19), 2007.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning

General Terms:
Algorithms, Design

Keywords:
collaborative filtering, graphical model, kalman filtering, matrix factorization, spatial modeling