Collaborative prediction using ensembles of Maximum Margin Matrix Factorizations

ABSTRACT

Fast gradient-based methods for Maximum Margin Matrix Factorization (MMMF) were recently shown to have great promise (Rennie & Srebro, 2005), including significantly outperforming the previous state-of-the-art methods on some standard collaborative prediction benchmarks (including MovieLens). In this paper, we investigate ways to further improve the performance of MMMF, by casting it within an ensemble approach. We explore and evaluate a variety of alternative ways to define such ensembles. We show that our resulting ensembles can perform significantly better than a single MMMF model, along multiple evaluation metrics. In fact, we find that ensembles of partially trained MMMF models can sometimes even give better predictions in total training time comparable to a single MMMF model.

REFERENCES

• 1: Daniel Billsus , Michael J. Pazzani, Learning Collaborative Information Filters, Proceedings of the Fifteenth International Conference on Machine Learning, p.46-54, July 24-27, 1998
• 2: Claypool, M., Gokhale, A., Miranda, T., Murnikov, P., Netes, D., & Sartin, M. (1999). Combining content-based and collaborative filters in an online newspaper. ACM SIGIR Workshop on Recommender Systems.
• 3: Derbeko, P., El-Yaniv, R., & Meir, R. (2002). Variance optimized bagging.
• 4: Nathaniel Good , J. Ben Schafer , Joseph A. Konstan , Al Borchers , Badrul Sarwar , Jon Herlocker , John Riedl, Combining collaborative filtering with personal agents for better recommendations, Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference innovative applications of artificial intelligence, p.439-446, July 18-22, 1999, Orlando, Florida, United States
• 5: Harrington, E., Herbrich, R., Kivinen, J., Platt, J. C., & Williamson, R. C. (2003). Online bayes point machines. Seventh Pacific-Asia Conference on Knowledge Discovery and Data Mining (pp. 241--252).
• 6: Thomas Hofmann, Latent semantic models for collaborative filtering, ACM Transactions on Information Systems (TOIS), v.22 n.1, p.89-115, January 2004 [doi>10.1145/963770.963774]
• 7: Lee, D., & Seung, H. (1999). Learning the parts of objects by non-negative matrix factorization. Nature, 401, 788--791.
• 8: Marlin, B. (2004). Collaborative filtering: A machine learning perspective. Master's thesis, University of Toronto.
• 9: Benjamin Marlin , Richard S. Zemel, The multiple multiplicative factor model for collaborative filtering, Proceedings of the twenty-first international conference on Machine learning, p.73, July 04-08, 2004, Banff, Alberta, Canada [doi>10.1145/1015330.1015437]
• 10: Prem Melville , Raymod J. Mooney , Ramadass Nagarajan, Content-boosted collaborative filtering for improved recommendations, Eighteenth national conference on Artificial intelligence, p.187-192, July 28-August 01, 2002, Edmonton, Alberta, Canada
• 11: Platt, J. C. (1999). Probabilities for support vector machines. In B. S. D. S. A. Smola (Ed.), Advances in large margin classifiers, 61--74. MIT Press.
• 12: Rennie, J. D. M. (2006). Personal Communication.
• 13: Jasson D. M. Rennie , Nathan Srebro, Fast maximum margin matrix factorization for collaborative prediction, Proceedings of the 22nd international conference on Machine learning, p.713-719, August 07-11, 2005, Bonn, Germany [doi>10.1145/1102351.1102441]
• 14: Srebro, N., Rennie, J. D. M., & Jaakola, T. S. (2005). Maximum-margin matrix factorization. NIPS.
• 15: Giorgio Valentini , Thomas G. Dietterich, Bias-Variance Analysis of Support Vector Machines for the Development of SVM-Based Ensemble Methods, The Journal of Machine Learning Research, 5, p.725-775, 12/1/2004

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.1 Models
          Subjects: Statistical

Additional Classification:
  F. Theory of Computation
  F.2 ANALYSIS OF ALGORITHMS AND PROBLEM COMPLEXITY
      F.2.1 Numerical Algorithms and Problems
          Subjects: Computations on matrices

General Terms:
Algorithms, Theory