Removing redundancy and inconsistency in memory-based collaborative filtering

ABSTRACT

The application range of memory-based collaborative filtering (CF) is limited due to CF's high memory consumption and long runtime. The approach presented in this paper removes redundant and inconsistent instances (users) from the data. This paper aims to distinguish informative instances (users) from large raw user preference database and thus alleviate the memory and runtime cost of the widely used memory-based collaborative filtering (CF) algorithm. Our work shows that a satisfactory accuracy can be achieved by using only a small portion of the original data set, thereby alleviating the storage and runtime cost of the CF algorithm. In our approach, we consider instance selection as the problem of selecting informative data that increase the We begin by discussing the instance selection problem in a general sense that is to increase a posteriori probability of the optimal model by selecting informative data. We evaluate the empirical performance of our approach PF on two real-world data sets and attain very promisingpositive experimental results. The dData size and the prediction time are significantly reduced, while the prediction accuracy is on a par with almost the same as the results achieved by using the complete database.

REFERENCES

• 1: David W. Aha , Dennis Kibler , Marc K. Albert, Instance-Based Learning Algorithms, Machine Learning, v.6 n.1, p.37-66, Jan. 1991 [doi>10.1023/A:1022689900470]
• 2: Daniel Billsus , Michael J. Pazzani, Learning Collaborative Information Filters, Proceedings of the Fifteenth International Conference on Machine Learning, p.46-54, July 24-27, 1998
• 3: J. S. Breese, D. Heckerman, and C. Kadie, "Empirical Analysis of Predictive Algorithms for Collaborative Filtering", In Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence, 1998.
• 4: D. Heckerman, "A Tutorial on Learning with Bayesian Networks", Technical Report MSR-TR-95-06, Microsoft Research, 1995.
• 5: S. Hettich and S. D. Bay. The UCI KDD Archive {http://kdd.ics.uci.edu}. Irvine, CA: University of California, Department of Information and Computer Science, (1999).
• 6: Huan Liu , Hiroshi Motoda, Instance Selection and Construction for Data Mining, Kluwer Academic Publishers, Norwell, MA, 2001
• 7: T Michell, "Machine Learning", McGraw Hill, 1997, pp169, 201--227
• 8: David M. Pennock , Eric Horvitz , Steve Lawrence , C. Lee Giles, Collaborative Filtering by Personality Diagnosis: A Hybrid Memory and Model-Based Approach, Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, p.473-480, June 30-July 03, 2000
• 9: Paul Resnick , Neophytos Iacovou , Mitesh Suchak , Peter Bergstrom , John Riedl, GroupLens: an open architecture for collaborative filtering of netnews, Proceedings of the 1994 ACM conference on Computer supported cooperative work, p.175-186, October 22-26, 1994, Chapel Hill, North Carolina, United States [doi>10.1145/192844.192905]
• 10: Upendra Shardanand , Pattie Maes, Social information filtering: algorithms for automating “word of mouth”, Proceedings of the SIGCHI conference on Human factors in computing systems, p.210-217, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223931]
• 11: D. Randall Wilson , Tony R. Martinez, Reduction Techniques for Instance-BasedLearning Algorithms, Machine Learning, v.38 n.3, p.257-286, March 2000 [doi>10.1023/A:1007626913721]
• 12: K. Yu, X. Xu, J. Tao, M. Ester, H.-P. Kriegel, "Instance Selection Techniques for Memory-Based Collaborative Filtering", Proc. 2nd SIAM Int. Conf. on Data Mining (SDM'02), 2002
• 13: Jianping Zhang, Selecting typical instances in instance-based learning, Proceedings of the ninth international workshop on Machine learning, p.470-479, July 1992, Aberdeen, Scotland, United Kingdom
• 14: Tong Zhang , Vijay S. Iyengar, Recommender systems using linear classifiers, The Journal of Machine Learning Research, 2, p.313-334, 3/1/2002 [doi>10.1162/153244302760200641]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces
          Subjects: Computer-supported cooperative work

General Terms:
Algorithms, Theory

Keywords:
collaborative filtering, data selection, efficiency, scalability