Avoiding monotony: improving the diversity of recommendation lists

ABSTRACT

The primary premise upon which top-N recommender systems operate is that similar users are likely to have similar tastes with regard to their product choices. For this reason, recommender algorithms depend deeply on similarity metrics to build the recommendation lists for end-users.

However, it has been noted that the products offered on recommendation lists are often too similar to each other and attention has been paid towards the goal of improving diversity to avoid monotonous recommendations.

Noting that the retrieval of a set of items matching a user query is a common problem across many applications of information retrieval, we model the competing goals of maximizing the diversity of the retrieved list while maintaining adequate similarity to the user query as a binary optimization problem. We explore a solution strategy to this optimization problem by relaxing it to a trust-region problem.This leads to a parameterized eigenvalue problem whose solution is finally quantized to the required binary solution. We apply this approach to the top-N prediction problem, evaluate the system performance on the Movielens dataset and compare it with a standard item-based top-N algorithm. A new evaluation metric ItemNovelty is proposed in this work. Improvements on both diversity and accuracy are obtained compared to the benchmark algorithm.

REFERENCES

• 1: Daniel M. Fleder , Kartik Hosanagar, Recommender systems and their impact on sales diversity, Proceedings of the 8th ACM conference on Electronic commerce, June 11-15, 2007, San Diego, California, USA [doi>10.1145/1250910.1250939]
• 2: Jonathan L. Herlocker , Joseph A. Konstan , Loren G. Terveen , John T. Riedl, Evaluating collaborative filtering recommender systems, ACM Transactions on Information Systems (TOIS), v.22 n.1, p.5-53, January 2004 [doi>10.1145/963770.963772]
• 3: George Karypis, Evaluation of Item-Based Top-N Recommendation Algorithms, Proceedings of the tenth international conference on Information and knowledge management, October 05-10, 2001, Atlanta, Georgia, USA [doi>10.1145/502585.502627]
• 4: Christopher D. Manning , Prabhakar Raghavan , Hinrich Schtze, Introduction to Information Retrieval, Cambridge University Press, New York, NY, 2008
• 5: L. McGinty and B. Smyth. On the role of diversity in conversational recommender systems. In Workshop Proceedings of the Fifth International Conference on Case-Based Reasoning(ICCBR 2003), page 1065, 2003.
• 6: David McSherry, Diversity-Conscious Retrieval, Proceedings of the 6th European Conference on Advances in Case-Based Reasoning, p.219-233, September 04-07, 2002
• 7: Bhaskar Mehta , Thomas Hofmann , Peter Fankhauser, Lies and propaganda: detecting spam users in collaborative filtering, Proceedings of the 12th international conference on Intelligent user interfaces, January 28-31, 2007, Honolulu, Hawaii, USA [doi>10.1145/1216295.1216307]
• 8: K. Nehring and C. Puppe. A theory of diversity. Econometrica, 70(3):1155--1198, May 2002.
• 9: C. Olsson, A. P. Eriksson, and F. Kahl. Solving large scale binary quadratic problems: Spectral methods vs. semidefinite programming. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition(CVPR '07), pages 1--8, 2007.
• 10: G. P. Patil and C. Taillie. Diversity as a concept and its measurement. Journal of the American Statistical Association, 77(379):548--561, Sep. 1982.
• 11: D. Reynolds. Comparison of background normalization methods for text-independent speaker verification. In Fifth European Conference on Speech Communication and Technology, 1997.
• 12: Marielba Rojas , Sandra A. Santos , Danny C. Sorensen, A New Matrix-Free Algorithm for the Large-Scale Trust-Region Subproblem, SIAM Journal on Optimization, v.11 n.3, p.611-646, 2000 [doi>10.1137/S105262349928887X]
• 13: Marielba Rojas , Sandra A. Santos , Danny C. Sorensen, Algorithm 873: LSTRS: MATLAB software for large-scale trust-region subproblems and regularization, ACM Transactions on Mathematical Software (TOMS), v.34 n.2, p.1-28, March 2008 [doi>10.1145/1326548.1326553]
• 14: Sean M. McNee , John Riedl , Joseph A. Konstan, Being accurate is not enough: how accuracy metrics have hurt recommender systems, CHI '06 extended abstracts on Human factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1125451.1125659]
• 15: Barry Smyth , Paul McClave, Similarity vs. Diversity, Proceedings of the 4th International Conference on Case-Based Reasoning: Case-Based Reasoning Research and Development, p.347-361, August 02, 2001
• 16: L. N. Vicente, J. J. Judice, and P. M. Pardalos. Parametric linear programming techniques for the indefinite quadratic programming problem. IMA Journal of Management Mathematics, 4(4):343--349, 1992.
• 17: Cai-Nicolas Ziegler , Sean M. McNee , Joseph A. Konstan , Georg Lausen, Improving recommendation lists through topic diversification, Proceedings of the 14th international conference on World Wide Web, May 10-14, 2005, Chiba, Japan [doi>10.1145/1060745.1060754]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

Additional Classification:
  H. Information Systems
  H.m MISCELLANEOUS

General Terms:
Algorithms, Performance, Theory

Keywords:
accuracy, diversity, metrics, novelty, recommender system