Selectively acquiring ratings for product recommendation

ABSTRACT

Accurate prediction of customer preferences on products is the key to any recommender systems to realize its promised strategic values such as improved customer satisfaction and therefore enhanced loyalty. In this paper, we propose proactively acquiring ratings from customers for a newly introduced product to quickly improve the accuracy of the predicted ratings generated by a collaborative filtering recommendation algorithm for the entire customer population. We formally introduce the problem of identifying the most informative ratings to acquire and termed it as the product rating acquisition problem. We proposed an active learning sampling method for this problem that is generic to any recommendation algorithms. Using the Netflix Prize dataset, we experimented with our proposed method, a uniform random sampling method, and a degree-based sampling method that is biased toward customers with large numbers of ratings for the user-based and item-based neighborhood recommendation algorithms. The experimental results showed that even with the random sampling method, acquiring 10% of all ratings in addition to a randomly selected 10% initial ratings achieved 4.5% improvement on overall rating prediction accuracy of the movie. In addition, our proposed active learning sampling method consistently outperformed the random and degree-based sampling for the better-performing item-based algorithm and achieved more than 8% improvement by acquiring 10% of the ratings.

REFERENCES

• 1: Gediminas Adomavicius , Alexander Tuzhilin, Using Data Mining Methods to Build Customer Profiles, Computer, v.34 n.2, p.74-82, February 2001
• 2: Dana Angluin, Queries and Concept Learning, Machine Learning, v.2 n.4, p.319-342, April 1988 [doi>10.1023/A:1022821128753]
• 3: Atkinson, A. The usefulness of optimum experimental designs. Journal of the Royal Statistical Society, 58. 59--76, 1996.
• 4: Bennett, J. and Lanning, S. The Netflix Prize, Netflix, 2007.
• 5: Boutilier, C., Zemel, R. S. and Marlin, B., Active collaborative filtering. in Nineteenth Annual Conference on Uncertainty in Artificial Intelligence, (2003), 98--106.
• 6: Breese, J. S., Heckerman, D. and Kadie, C., Empirical analysis of predictive algorithms for collaborative filtering. in Fourteenth Conference on Uncertainty in Artificial Intelligence, (Madison, WI, 1998), Morgan Kaufmann, 43--52.
• 7: David Cohn , Les Atlas , Richard Ladner, Improving Generalization with Active Learning, Machine Learning, v.15 n.2, p.201-221, May 1994 [doi>10.1023/A:1022673506211]
• 8: Cohn, D. A., Ghahramani, Z. and Jordan, M. I. Active learning with statistical models. Journal of Artificial Intelligence Research, 4. 129--145, 1996.
• 9: Mukund Deshpande , George Karypis, Item-based top-N recommendation algorithms, ACM Transactions on Information Systems (TOIS), v.22 n.1, p.143-177, January 2004 [doi>10.1145/963770.963776]
• 10: Yoav Freund , H. Sebastian Seung , Eli Shamir , Naftali Tishby, Selective Sampling Using the Query by Committee Algorithm, Machine Learning, v.28 n.2-3, p.133-168, Aug./Sept. 1997 [doi>10.1023/A:1007330508534]
• 11: Ken Goldberg , Theresa Roeder , Dhruv Gupta , Chris Perkins, Eigentaste: A Constant Time Collaborative Filtering Algorithm, Information Retrieval, v.4 n.2, p.133-151, July 2001 [doi>10.1023/A:1011419012209]
• 12: Thomas Hofmann, Latent semantic models for collaborative filtering, ACM Transactions on Information Systems (TOIS), v.22 n.1, p.89-115, January 2004 [doi>10.1145/963770.963774]
• 13: Thomas Hofmann , Jan Puzicha, Latent Class Models for Collaborative Filtering, Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, p.688-693, July 31-August 06, 1999
• 14: Zan Huang , Hsinchun Chen , Daniel Zeng, Applying associative retrieval techniques to alleviate the sparsity problem in collaborative filtering, ACM Transactions on Information Systems (TOIS), v.22 n.1, p.116-142, January 2004 [doi>10.1145/963770.963775]
• 15: Huang, Z. and Zeng, D., Why does collaborative filtering work?- Recommendation model validation and selection by analyzing random bipartite graphs. in Fifteenth Annual Workshop on Information Technologies and Systems (WITS 2005), (Las Vegas, NV, 2005).
• 16: Zan Huang , Daniel Zeng , Hsinchun Chen, A Comparison of Collaborative-Filtering Recommendation Algorithms for E-commerce, IEEE Intelligent Systems, v.22 n.5, p.68-78, September 2007 [doi>10.1109/MIS.2007.80]
• 17: Huang, Z., Zeng, D. and Chen, H., A link analysis approach to recommendation with sparse data. in Americas Conference on Information Systems, (New York, NY, 2004), 1997--2005.
• 18: Rong Jin , Luo Si, A Bayesian approach toward active learning for collaborative filtering, Proceedings of the 20th conference on Uncertainty in artificial intelligence, p.278-285, July 07-11, 2004, Banff, Canada
• 19: Kiefer, J. Optimal experimental designs. Journal of the Royal Statistical Society, series B, 21. 272--304, 1959.
• 20: Lewis, D. and Gale, W., Training text classifiers by uncertainty sampling. in International ACM Conference on Research and Development in Information Retrieval (SIGIR-94), (1994), 3--12.
• 21: Weiyang Lin , Sergio A. Alvarez , Carolina Ruiz, Efficient Adaptive-Support Association Rule Mining for Recommender Systems, Data Mining and Knowledge Discovery, v.6 n.1, p.83-105, January 2002 [doi>10.1023/A:1013284820704]
• 22: Greg Linden , Brent Smith , Jeremy York, Amazon.com Recommendations: Item-to-Item Collaborative Filtering, IEEE Internet Computing, v.7 n.1, p.76-80, January 2003 [doi>10.1109/MIC.2003.1167344]
• 23: Michael J. Pazzani, A Framework for Collaborative, Content-Based and Demographic Filtering, Artificial Intelligence Review, v.13 n.5-6, p.393-408, Dec. 1999 [doi>10.1023/A:1006544522159]
• 24: David M. Pennock , Eric Horvitz , Steve Lawrence , C. Lee Giles, Collaborative Filtering by Personality Diagnosis: A Hybrid Memory and Model-Based Approach, Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, p.473-480, June 30-July 03, 2000
• 25: Alexandrin Popescul , Lyle H. Ungar , David M. Pennock , Steve Lawrence, Probabilistic Models for Unified Collaborative and Content-Based Recommendation in Sparse-Data Environments, Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence, p.437-444, August 02-05, 2001
• 26: Paul Resnick , Neophytos Iacovou , Mitesh Suchak , Peter Bergstrom , John Riedl, GroupLens: an open architecture for collaborative filtering of netnews, Proceedings of the 1994 ACM conference on Computer supported cooperative work, p.175-186, October 22-26, 1994, Chapel Hill, North Carolina, United States [doi>10.1145/192844.192905]
• 27: Robbins, H. Some aspects of the sequential design of experiments. Bulletin American Mathematical Society, 55. 527--535, 1952.
• 28: Nicholas Roy , Andrew McCallum, Toward Optimal Active Learning through Sampling Estimation of Error Reduction, Proceedings of the Eighteenth International Conference on Machine Learning, p.441-448, June 28-July 01, 2001
• 29: Maytal Saar-Tsechansky , Foster Provost, Active Sampling for Class Probability Estimation and Ranking, Machine Learning, v.54 n.2, p.153-178, February 2004 [doi>10.1023/B:MACH.0000011806.12374.c3]
• 30: Saar-Tsechansky, M. and Provost, F. Decision-centric Active Learning of Binary-Outcome Models. Information Systems Research, forthcoming, 2007.
• 31: Sarwar, B., Karypis, G., Konstan, J. and Riedl, J., Application of dimensionality reduction in recommender systems: a case study. in WebKDD Workshop at the ACM SIGKKD, (Boston, MA, 2000).
• 32: Badrul Sarwar , George Karypis , Joseph Konstan , John Reidl, Item-based collaborative filtering recommendation algorithms, Proceedings of the 10th international conference on World Wide Web, p.285-295, May 01-05, 2001, Hong Kong, Hong Kong [doi>10.1145/371920.372071]
• 33: J. Ben Schafer , Joseph A. Konstan , John Riedl, E-Commerce Recommendation Applications, Data Mining and Knowledge Discovery, v.5 n.1-2, p.115-153, January-April 2001 [doi>10.1023/A:1009804230409]
• 34: Si, L. and Jin, R., Flexible mixture model for collaborative filtering. in Twentieth International Conference on Machine Learning, (2003).
• 35: Ungar, L. H. and Foster, D. P., A formal statistical approach to collaborative filtering. in Conference on Automated Learning and Discovery (CONALD), (Pittsburgh, PA, 1998).
• 36: Kai Yu , Anton Schwaighofer , Volker Tresp , Xiaowei Xu , Hans-Peter Kriegel, Probabilistic Memory-Based Collaborative Filtering, IEEE Transactions on Knowledge and Data Engineering, v.16 n.1, p.56-69, January 2004 [doi>10.1109/TKDE.2004.1264822]
• 37: Zhiqiang Zheng , Balaji Padmanabhan, Selectively Acquiring Customer Information: A New Data Acquisition Problem and an Active Learning-Based Solution, Management Science, v.52 n.5, p.697-712, May 2006 [doi>10.1287/mnsc.1050.0488]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human information processing

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models; Relevance feedback; Information filtering

General Terms:
Algorithms, Design, Experimentation

Keywords:
active learning, collaborative filtering, recommender systems, sampling