Evaluating peer-to-peer recommender systems that exploit spontaneous affinities

ABSTRACT

The validation of a recommender system is always a quite hazardous task, because of the difficulty of modeling the tastes of a given user. Novel (decentralized) recommender systems are proposed and evaluated by way of well known logs of user profiles and buddy tables, that contain lists of items with feedback ratings assigned by a given set of users. These information are cross linked, and the precision of the recommendation is compared with other well known (centralized) systems. This evaluation approach cannot be applied in the actual peer-to-peer domain: it is difficult, if not impossible, to build and maintain user profiles, and users are not required to give feedbacks to a data collector entity. Moreover, objects are poorly or not structured, and meta-information, when present, cannot be trusted because of fake files and incomplete item descriptions.

In this paper, we present an evaluation process based on a 10-fold cross validation task, that we applied to estimate accuracy of the suggestions of a P2P recommender system recently proposed in [2]. The complexity of the evaluation of this peculiar recommender is increased because of "spontaneous affinities" between users that are used instead of classical knowledge representation based strategies.

REFERENCES

• 1: I. F. A. Iamnitchi, M. Ripeanu. Small-world file-sharing communities. In The 23rd Conference of the IEEE Communications Society (InfoCom 2004), Hong Kong, 2004.
• 2: G. Ruffo, R. Schifanella, E. Ghiringhello A Decentralized Recommendation System based on Self-Organizing Partnerships In IFIP-Networking'06, May 2006, LNCS 3976:618--629, Coimbra (Portugal), 2006.
• 3: Jun Wang , Johan Pouwelse , Reginald L. Lagendijk , Marcel J. T. Reinders, Distributed collaborative filtering for peer-to-peer file sharing systems, Proceedings of the 2006 ACM symposium on Applied computing, April 23-27, 2006, Dijon, France [doi>10.1145/1141277.1141522]
• 4: B. Krulwich. Lifestyle finder: Intelligent user profiling using large-scale demographic data. AI Magazine, 18(2):37--45, 1997.
• 5: K. Lang. NewsWeeder: learning to filter netnews. In Proc. of the 12th ICML, pages 331--339. Morgan Kaufmann publishers Inc.: San Mateo, CA, USA, 1995.
• 6: M. E. J. Newman. The structure and function of complex networks. SIAM Review, 45:167, 2003.
• 7: M. J. Pazzani, J. Muramatsu, and D. Billsus. Syskill webert: Identifying interesting web sites. In AAAI/IAAI, Vol. 1, pages 54--61, 1996.
• 8: Alexandrin Popescul , Lyle H. Ungar , David M. Pennock , Steve Lawrence, Probabilistic Models for Unified Collaborative and Content-Based Recommendation in Sparse-Data Environments, Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence, p.437-444, August 02-05, 2001
• 9: Paul Resnick , Hal R. Varian, Recommender systems, Communications of the ACM, v.40 n.3, p.56-58, March 1997 [doi>10.1145/245108.245121]
• 10: Amund Tveit, Peer-to-peer based recommendations for mobile commerce, Proceedings of the 1st international workshop on Mobile commerce, p.26-29, July 2001, Rome, Italy [doi>10.1145/381461.381466]
• 11: D. J. Watts and S. H. Strogatz. Collective dynamics of 'small-world' networks. Nature, 393(6684):440--442, June 1998.
• 12: Yan Zheng Wei , Luc Moreau , Nicholas R. Jennings, A market-based approach to recommender systems, ACM Transactions on Information Systems (TOIS), v.23 n.3, p.227-266, July 2005 [doi>10.1145/1080343.1080344]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Information filtering

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.4 Distributed Systems
          Subjects: Distributed applications

General Terms:
Algorithms, Design, Human Factors, Measurement

Keywords:
complex and social networks, file sharing systems, peer-to-peer, recommender system