'Helpfulness' in online communities: a measure of message quality

ABSTRACT

Online communities displaying textual postings require measures to combat information overload. One popular approach is to ask participants whether or not messages are helpful in order to then guide others to interesting content. Adopting a well-established framework for assessing data quality, we examine the nature of "helpfulness."We study consumer reviews at Amazon.com, deriving 22 measures quantifying their textual properties, authors' reputations and product characteristics. Confirmatory factor analysis reveals five underlying quality dimensions representing reviewers' reputations in the community, the topical relevancy of the reviews, the ease of understanding them, their believability and objectivity. A correlation and regression analysis confirms that these dimensions are related to the helpfulness scores assigned by community participants. However, it also uncovers a strong relationship between the chronological ordering of reviews and helpfulness, which both community participants and designers should keep in mind when using this method of social navigation.

REFERENCES

• 1: Chevalier, J.A. and Mayzlin, D. The effect of word of mouth on sales: Online book reviews. Journal of Marketing Research 43, 3 (2006), 345--354.
• 2: David, S. and Pinch, T. Six degrees of reputation: the use and abuse of online review and recommendation systems. First Monday 11, 3 (2006).
• 3: Foltz, P.W., Laham, D., Landauer, T.K. Automated essay scoring: Applications to educational technology. In Proc. EdMedia, (1999).
• 4: Anindya Ghose , Panagiotis G. Ipeirotis, Designing novel review ranking systems: predicting the usefulness and impact of reviews, Proceedings of the ninth international conference on Electronic commerce, August 19-22, 2007, Minneapolis, MN, USA [doi>10.1145/1282100.1282158]
• 5: David Goldberg , David Nichols , Brian M. Oki , Douglas Terry, Using collaborative filtering to weave an information tapestry, Communications of the ACM, v.35 n.12, p.61-70, Dec. 1992 [doi>10.1145/138859.138867]
• 6: Gorsuch, R.L. Factor Analysis. Lawrence Erlbaum, Hillsdale, New Jersey, 1983.
• 7: Bin Gu , Prabhudev Konana , Balaji Rajagopalan , Hsuan-Wei Michelle Chen, Competition Among Virtual Communities and User Valuation: The Case of Investing-Related Communities, Information Systems Research, v.18 n.1, p.68-85, March 2007 [doi>10.1287/isre.1070.0114]
• 8: Starr R. Hiltz , Murray Turoff, Structuring computer-mediated communication systems to avoid information overload, Communications of the ACM, v.28 n.7, p.680-689, July 1985 [doi>10.1145/3894.3895]
• 9: Bernard J. Jansen , Amanda Spink , Tefko Saracevic, Real life, real users, and real needs: a study and analysis of user queries on the web, Information Processing and Management: an International Journal, v.36 n.2, p.207-227, Jan.1.2000 [doi>10.1016/S0306-4573(99)00056-4]
• 10: Quentin Jones , Gilad Ravid , Sheizaf Rafaeli, Information Overload and the Message Dynamics of Online Interaction Spaces: A Theoretical Model and Empirical Exploration, Information Systems Research, v.15 n.2, p.194-210, June 2004 [doi>10.1287/isre.1040.0023]
• 11: Kaiser, H.F. The application of electronic computers to factor analysis. Educational and Psychological Measurement 20, (1960), 141--151.
• 12: Cliff A.C. Lampe , Erik Johnston , Paul Resnick, Follow the reader: filtering comments on slashdot, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240815]
• 13: Loehlin, J. C. Latent Variable Models. Lawrence Erlbaum Associates, 1992.
• 14: Christopher D. Manning , Hinrich Schütze, Foundations of statistical natural language processing, MIT Press, Cambridge, MA, 1999
• 15: Otterbacher, J. Managing information in online product review communities: a comparison of two approaches. In Proc. 16th European Conference on Information Systems (ECIS 2008), Association for Information Systems (2008).
• 16: Leo L. Pipino , Yang W. Lee , Richard Y. Wang, Data quality assessment, Communications of the ACM, v.45 n.4, April 2002 [doi>10.1145/505248.506010]
• 17: Dragomir R. Radev , Hongyan Jing , Małgorzata Styś , Daniel Tam, Centroid-based summarization of multiple documents, Information Processing and Management: an International Journal, v.40 n.6, p.919-938, November 2004 [doi>10.1016/j.ipm.2003.10.006]
• 18: Al M. Rashid , Kimberly Ling , Regina D. Tassone , Paul Resnick , Robert Kraut , John Riedl, Motivating participation by displaying the value of contribution, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124915]
• 19: Rindova, V., Petkova, A.P. and Kotha, S. Standing out: How new firms in emerging marketing build reputation and knowledge creation. Strategic Organization 5, 31 (2007), 31--70.
• 20: Warren Sack, Conversation Map: An Interface for Very Large-Scale Conversations, Journal of Management Information Systems, v.17 n.3, p.73-92, Number 3/Winter 2000/01
• 21: Gerard Salton , Michael J. McGill, Introduction to Modern Information Retrieval, McGraw-Hill, Inc., New York, NY, 1986
• 22: Schindler, R.M. and Bickart, B. Published word of mouth: Referable, consumer-generated information on the Internet. In: Hauvgedt, C., Machleit, K. and Yalch, R. (eds.) Online Consumer Psychology: Understanding and Influencing Behavior in the Virtual World. Lawrence Erlbaum Associates, 2005, 35--61.
• 23: Rong Tang , Kwong Bor Ng , Tomek Strzalkowski , Paul B. Kantor, Automatically predicting information quality in news documents, Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: companion volume of the Proceedings of HLT-NAACL 2003--short papers, p.97-99, May 27-June 01, 2003, Edmonton, Canada [doi>10.3115/1073483.1073516]
• 24: Richard Y. Wang , Diane M. Strong, Beyond accuracy: what data quality means to data consumers, Journal of Management Information Systems, v.12 n.4, p.5-33, March 1996
• 25: Zhu Zhang , Balaji Varadarajan, Utility scoring of product reviews, Proceedings of the 15th ACM international conference on Information and knowledge management, November 06-11, 2006, Arlington, Virginia, USA [doi>10.1145/1183614.1183626]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces

General Terms:
Design, Human Factors

Keywords:
information overload, information quality, online community, product reviews, social navigation