How opinions are received by online communities: a case study on amazon.com helpfulness votes

ABSTRACT

There are many on-line settings in which users publicly express opinions. A number of these offer mechanisms for other users to evaluate these opinions; a canonical example is Amazon.com, where reviews come with annotations like "26 of 32 people found the following review helpful." Opinion evaluation appears in many off-line settings as well, including market research and political campaigns. Reasoning about the evaluation of an opinion is fundamentally different from reasoning about the opinion itself: rather than asking, "What did Y think of X?", we are asking, "What did Z think of Y's opinion of X?" Here we develop a framework for analyzing and modeling opinion evaluation, using a large-scale collection of Amazon book reviews as a dataset. We find that the perceived helpfulness of a review depends not just on its content but also but also in subtle ways on how the expressed evaluation relates to other evaluations of the same product. As part of our approach, we develop novel methods that take advantage of the phenomenon of review "plagiarism" to control for the effects of text in opinion evaluation, and we provide a simple and natural mathematical model consistent with our findings. Our analysis also allows us to distinguish among the predictions of competing theories from sociology and social psychology, and to discover unexpected differences in the collective opinion-evaluation behavior of user populations from ifferent countries.

REFERENCES

• 1: E. Agichtein, C. Castillo, D. Donato, A. Gionis, and G. Mishne. Finding high-quality content in social media. In procof WSDM, 2008.
• 2: A. Agresti. phAn Introduction to Categorial Data Analysis. Wiley, 1996.
• 3: T. Amabile. Brilliant but cruel: Perception of negative evaluators. Journal of Experimental Social Psychology, 19: 146--156, 1983.
• 4: {4}R. Bond and P. B. Smith. Culture and conformity: A meta-analysis of studies using Asch's (1952b, 1956) line judgment task. Psychological Bulletin, 119 (1): 111--137, 1996.
• 5: J. A. Chevalier and D. Mayzlin. The effect of word of mouth on sales: Online book reviews. Journal of Marketing Research, 43 (3): 345--354, August 2006.
• 6: R. B. Cialdini and N. J. Goldstein. Social influence: compliance and conformity. Annual Review of Psychology, 55: 591--621, 2004.
• 7: S. David and T. J. Pinch. Six degrees of reputation: The use and abuse of online review and recommendation systems. First Monday, July 2006. Special Issue on Commercial Applications of the Internet.
• 8: J. Escalas and J. R. Bettman. Self-construal, reference groups, and brand meaning. Journal of Consumer Research, 32 (3): 378--389, 2005.
• 9: G. J. Fitzsimons, J. W. Hutchinson, P. Williams, and J. W. Alba. Non-conscious influences on consumer choice. Marketing Letters, 13 (3): 267--277, 2002.
• 10: Anindya Ghose , Panagiotis G. Ipeirotis, Designing novel review ranking systems: predicting the usefulness and impact of reviews, Proceedings of the ninth international conference on Electronic commerce, August 19-22, 2007, Minneapolis, MN, USA [doi>10.1145/1282100.1282158]
• 11: A. Ghose and P. G. Ipeirotis. Estimating the socio-economic impact of product reviews: Mining text and reviewer characteristics. SSRN working paper, 2008. Available at http://ssrn.com/paper=1261751. Version dated September 1.
• 12: F. Maxwell Harper , Daphne Raban , Sheizaf Rafaeli , Joseph A. Konstan, Predictors of answer quality in online Q&A sites, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357191]
• 13: Nan Hu , Paul A. Pavlou , Jennifer Zhang, Can online reviews reveal a product's true quality?: empirical findings and analytical modeling of Online word-of-mouth communication, Proceedings of the 7th ACM conference on Electronic commerce, p.324-330, June 11-15, 2006, Ann Arbor, Michigan, USA [doi>10.1145/1134707.1134743]
• 14: Nitin Jindal , Bing Liu, Review spam detection, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242759]
• 15: S.-M. Kim, P. Pantel, T. Chklovski, and M. Pennacchiotti. Automatically assessing review helpfulness. In procof EMNLP, pages 423--430, July 2006.
• 16: B. Knutson, S. Rick, G. E. Wimmer, D. Prelec, and G. Loewenstein. Neural predictors of purchases. Neuron 53: 147--156, 2007.
• 17: J. Liu, Y. Cao, C.-Y. Lin, Y. Huang, and M. Zhou. Low-quality product review detection in opinion summarization. In procof EMNLP-CoNLL, pages 334--342, 2007. Poster paper.
• 18: Bo Pang , Lillian Lee, Opinion Mining and Sentiment Analysis, Foundations and Trends in Information Retrieval, v.2 n.1-2, p.1-135, January 2008 [doi>10.1561/1500000011]
• 19: S. Sen, F. M. Harper, A. LaPitz, and J. Riedl. The quest for quality tags. In procof GROUP, 2007.
• 20: Daria Sorokina , Johannes Gehrke , Simeon Warner , Paul Ginsparg, Plagiarism Detection in arXiv, Proceedings of the Sixth International Conference on Data Mining, p.1070-1075, December 18-22, 2006 [doi>10.1109/ICDM.2006.126]
• 21: Paco Underhill , Rick Adamson, Why We Buy: The Science of Shopping, RH Audio, 2007
• 22: H. T. Welser, E. Gleave, D. Fisher, and M. Smith. Visualizing the signatures of social roles in online discussion groups. Journal of Social Structure, 2007.
• 23: Fang Wu , Bernardo A. Huberman, How Public Opinion Forms, Proceedings of the 4th International Workshop on Internet and Network Economics, December 17-20, 2008, Shanghai, China [doi>10.1007/978-3-540-92185-1_39]
• 24: Zhu Zhang , Balaji Varadarajan, Utility scoring of product reviews, Proceedings of the 15th ACM international conference on Information and knowledge management, November 06-11, 2006, Arlington, Virginia, USA [doi>10.1145/1183614.1183626]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining

General Terms:
Measurement, Theory

Keywords:
online communities, opinion mining, plagiarism, review helpfulness, review utility, sentiment analysis, social influence