Is seeing believing?: how recommender system interfaces affect users' opinions

ABSTRACT

Recommender systems use people's opinions about items in an information domain to help people choose other items. These systems have succeeded in domains as diverse as movies, news articles, Web pages, and wines. The psychological literature on conformity suggests that in the course of helping people make choices, these systems probably affect users' opinions of the items. If opinions are influenced by recommendations, they might be less valuable for making recommendations for other users. Further, manipulators who seek to make the system generate artificially high or low recommendations might benefit if their efforts influence users to change the opinions they contribute to the recommender. We study two aspects of recommender system interfaces that may affect users' opinions: the rating scale and the display of predictions at the time users rate items. We find that users rate fairly consistently across rating scales. Users can be manipulated, though, tending to rate toward the prediction the system shows, whether the prediction is accurate or not. However, users can detect systems that manipulate predictions. We discuss how designers of recommender systems might react to these findings.

REFERENCES

• 1: T. Amoo and H. H. Friedman. Do numeric values influence subjects' responses to rating scales? Journal of International Marketing and Marketing Research, 26:41--46, February 2001.
• 2: S. E. Asch. Effects of group pressure upon the modification and distortion of judgements. In Groups, Leadership, and Men, pages 177--190, 1951.
• 3: J. S. Breese, D. Heckerman, and C. Kadie. Empirical analysis of predictive algorithms for collaborative filtering. In Proc. UAI '98, pages 43--52, July 1998.
• 4: Chrysanthos Dellarocas, Immunizing online reputation reporting systems against unfair ratings and discriminatory behavior, Proceedings of the 2nd ACM conference on Electronic commerce, p.150-157, October 17-20, 2000, Minneapolis, Minnesota, United States [doi>10.1145/352871.352889]
• 5: Pedro Domingos , Matt Richardson, Mining the network value of customers, Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining, p.57-66, August 26-29, 2001, San Francisco, California [doi>10.1145/502512.502525]
• 6: H. H. Friedman and T. Amoo. Rating the rating scales. Journal of Marketing Management, 9(3):114--123, 1999.
• 7: R. Garland. The midpoint on a rating scale: Is it desirable? Marketing Bulletin, 2:66--70, 1991.
• 8: Ken Goldberg , Theresa Roeder , Dhruv Gupta , Chris Perkins, Eigentaste: A Constant Time Collaborative Filtering Algorithm, Information Retrieval, v.4 n.2, p.133-151, July 2001 [doi>10.1023/A:1011419012209]
• 9: Jonathan L. Herlocker , Joseph A. Konstan , John Riedl, Explaining collaborative filtering recommendations, Proceedings of the 2000 ACM conference on Computer supported cooperative work, p.241-250, December 2000, Philadelphia, Pennsylvania, United States [doi>10.1145/358916.358995]
• 10: Will Hill , Larry Stead , Mark Rosenstein , George Furnas, Recommending and evaluating choices in a virtual community of use, Proceedings of the SIGCHI conference on Human factors in computing systems, p.194-201, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223929]
• 11: C. Nass and Y. Moon. Machines and mindlessness: Social responses to computers. Journal of Social Issues, 60(1):81--103, 2000.
• 12: David M. Pennock , Eric Horvitz , Steve Lawrence , C. Lee Giles, Collaborative Filtering by Personality Diagnosis: A Hybrid Memory and Model-Based Approach, Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, p.473-480, June 30-July 03, 2000
• 13: Paul Resnick , Neophytos Iacovou , Mitesh Suchak , Peter Bergstrom , John Riedl, GroupLens: an open architecture for collaborative filtering of netnews, Proceedings of the 1994 ACM conference on Computer supported cooperative work, p.175-186, October 22-26, 1994, Chapel Hill, North Carolina, United States [doi>10.1145/192844.192905]
• 14: Badrul Sarwar , George Karypis , Joseph Konstan , John Reidl, Item-based collaborative filtering recommendation algorithms, Proceedings of the 10th international conference on World Wide Web, p.285-295, May 01-05, 2001, Hong Kong, Hong Kong [doi>10.1145/371920.372071]
• 15: Upendra Shardanand , Pattie Maes, Social information filtering: algorithms for automating “word of mouth”, Proceedings of the SIGCHI conference on Human factors in computing systems, p.210-217, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223931]
• 16: K. Swearingen and R. Sinha. Interaction design for recommender systems. In Designing Interactive Systems (DIS2002), London, June 25--28 2002.
• 17: Andrew H. Turpin , William Hersh, Why batch and user evaluations do not give the same results, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.225-231, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383992]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors; Software psychology
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Information filtering

General Terms:
Design, Experimentation, Human Factors

Keywords:
collaborative filtering, conformity, e-commerce, persuasive computing, recommender systems