Predictors of answer quality in online Q&A sites

ABSTRACT

Question and answer (Q&A) sites such as Yahoo! Answers are places where users ask questions and others answer them. In this paper, we investigate predictors of answer quality through a comparative, controlled field study of responses provided across several online Q&A sites. Along with several quantitative results concerning the effects of factors such as question topic and rhetorical strategy, we present two high-level messages. First, you get what you pay for in Q&A sites. Answer quality was typically higher in Google Answers (a fee-based site) than in the free sites we studied, and paying more money for an answer led to better outcomes. Second, we find that a Q&A site's community of users contributes to its success. Yahoo! Answers, a Q&A site where anybody can answer questions, outperformed sites that depend on specific individuals to answer questions, such as library reference services.

REFERENCES

• 1: Box, G., Hunter, W, Hunter, S., Hunter, W. Statistics for Experimenters. John Wiley & Sons, New York (1978).
• 2: Chae, M., Lee, B. Transforming an Online Portal Site Into a Playground for Netizen. J. of Internet Commerce 4, 2 (2005).
• 3: Deci, E, Koestner, R., Ryan, R. A Meta-Analytic Review of Experiments Examining the Effects of Extrinsic Rewards on Intrinsic Motivation. Psychological Bulletin 125, 6 (1999).
• 4: Edelman, B. Earnings and Ratings at Google Answers. Unpublished Manuscript (2004).
• 5: Gazan, R. Specialists and Synthesists in a Question Answering Community. In Proc. American Society for Information Science and Technology, 43 (2006).
• 6: Giles, J. Internet Encyclopaedias Go Head to Head. Nature 438 (2005), 900--901.
• 7: Joseph Janes , Chrystie Hill , Alex Rolfe, Ask-an-expert services analysis, Journal of the American Society for Information Science and Technology, v.52 n.13, p.1106-1121, November 2001 [doi>10.1002/asi.1177]
• 8: Cliff Lampe , Paul Resnick, Slash(dot) and burn: distributed moderation in a large online conversation space, Proceedings of the SIGCHI conference on Human factors in computing systems, p.543-550, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985761]
• 9: Lee, J. H., Downie, J. S., Cunningham, S. J. Challenges in Cross-Cultural/Multilingual Music Information Seeking. In Proc ISMIR (2005).
• 10: Ling, K., et al. Using social psychology to motivate contributions to online communities. Journal of Computer Mediated Communication 10, 4 (2005).
• 11: Leibenluft, J. A Librarian's Worst Nightmare: Yahoo! Answers, where 120 million users can be wrong. Slate Magazine, Dec. 7, 2007 (2007).
• 12: Mcclennen, M., Memmott, P. Roles in Digital Reference. Information Technology and Libraries 20, 3 (2001).
• 13: Jeffrey Pomerantz , Scott Nicholson , Yvonne Belanger , R. David Lankes, The current state of digital reference: validation of a general digital reference model through a survey of digital reference services, Information Processing and Management: an International Journal, v.40 n.2, p.347-363, March 2004 [doi>10.1016/S0306-4573(02)00085-7]
• 14: Reid Priedhorsky , Jilin Chen , Shyong (Tony) K. Lam , Katherine Panciera , Loren Terveen , John Riedl, Creating, destroying, and restoring value in wikipedia, Proceedings of the 2007 international ACM conference on Supporting group work, November 04-07, 2007, Sanibel Island, Florida, USA [doi>10.1145/1316624.1316663]
• 15: Raban, D., Harper, F. Motivations for Answering Questions Online. In New Media and Innovative Technologies (2008).
• 16: Rafaeli, S., Raban, D., Ravid, G. How Social Motivation Enhances Economic Activity and Incentives in the Google Answers Knowledge Sharing Market. Int. Journal of Knowledge and Learning 3, 1 (2007), 1--11.
• 17: Rafaeli, S., Raban, D., Ravid, G. Social and economic incentives in Google Answers. ACM Group 2005 Workshop: Sustaining Community: The role and design of incentive mechanisms in online systems (2005).
• 18: Rousch, W. What's the Best Q&A Site? Technology Review, December 3, 2006 (2006).
• 19: Sang-hun, C. South Koreans Connect Through Search Engine. New York Times, July 5, 2007 (2007).
• 20: White, M. Diffusion of an Innovation: Digital Reference Service in Carnegie Foundation Master's (Comprehensive) Academic Institution Libraries. Journal of Academic Librarianship 27, 3 (2001), 173--187.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces
          Subjects: Web-based interaction

General Terms:
Design, Experimentation, Human Factors

Keywords:
digital reference, expert services, information exchanges, information quality, knowledge networks, online community, q&a