Slash(dot) and burn: distributed moderation in a large online conversation space

ABSTRACT

Can a system of distributed moderation quickly and consistently separate high and low quality comments in an online conversation? Analysis of the site Slashdot.org suggests that the answer is a qualified yes, but that important challenges remain for designers of such systems. Thousands of users act as moderators. Final scores for comments are reasonably dispersed and the community generally agrees that moderations are fair. On the other hand, much of a conversation can pass before the best and worst comments are identified. Of those moderations that were judged unfair, only about half were subsequently counterbalanced by a moderation in the other direction. And comments with low scores, not at top-level, or posted late in a conversation were more likely to be overlooked by moderators.

REFERENCES

• 1: Avery, C., P. Resnick, and R. Zeckhauser, The Market for Evaluations. American Economic Review, 1999. 89(3): p. 564--584.
• 2: Banerjee, A., A Simple Model of Herd Behavior. Quarterly Journal of Economics, 1992. 107(3): p. 797--818.
• 3: Bikhchandani, S., D. Hirshleifer, and I. Welch, A Theory of Fads, Fashion, Custom and Cultural Change as Informational Cascades. Journal of Political Economy, 1989. 100(5): p. 992--1026.
• 4: Butler, B., When is a group not a group: An empirical examination of metaphors for online social structure. 1999, Carnegie Mellon University: Pittsburgh.
• 5: Friedman, E.J. and P. Resnick, The Social Cost of Cheap Pseudonyms. Journal of Economics and Management Strategy, 1997. 10(2): p. 173--179.
• 6: David Goldberg , David Nichols , Brian M. Oki , Douglas Terry, Using collaborative filtering to weave an information tapestry, Communications of the ACM, v.35 n.12, p.61-70, Dec. 1992 [doi>10.1145/138859.138867]
• 7: Grice, H.P., Utterer's meaning and intentions. Philosophical Review, 1969. 78: p. 147--177.
• 8: Q. Jones , G. Ravid , S. Rafaeli, An Empirical Exploration of Mass Interaction System Dynamics: Individual Information Overload and Usenet Discourse, Proceedings of the 35th Annual Hawaii International Conference on System Sciences (HICSS'02)-Volume 4, p.114.2, January 07-10, 2002
• 9: Kollock, P. and M. Smith, Managing the Virtual Commons: Cooperation and Conflict in Computer Communities, in Computer-Mediated Communication: Linguistic, Social, and Cross-Cultural Perspectives, S. Herring, Editor. 1996, John Benjamin: Amsterdam.
• 10: Malda, R., Slashdot FAQ, . 2003, http://slashdot.org
• 11: Pfaffenberger, B., A Standing Wave in the Web of Our Communications: Usenet and the Socio-Technical Construction of Cyberspace Values, in From Usenet to CoWebs: Interacting with Social Information Spaces, C. Lueg and D. Fisher, Editors. 2002, Springer Verlag: New York, NY.
• 12: Paul Resnick , Neophytos Iacovou , Mitesh Suchak , Peter Bergstrom , John Riedl, GroupLens: an open architecture for collaborative filtering of netnews, Proceedings of the 1994 ACM conference on Computer supported cooperative work, p.175-186, October 22-26, 1994, Chapel Hill, North Carolina, United States [doi>10.1145/192844.192905]
• 13: Sack, W., Conversation map: An interface for very large-scale conversations. Journal of Management information Systems, 2000. 17(3): p. 73--92.
• 14: Upendra Shardanand , Pattie Maes, Social information filtering: algorithms for automating “word of mouth”, Proceedings of the SIGCHI conference on Human factors in computing systems, p.210-217, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223931]
• 15: Marc A. Smith , Andrew T. Fiore, Visualization components for persistent conversations, Proceedings of the SIGCHI conference on Human factors in computing systems, p.136-143, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365073]
• 16: Terveen, L. and W. Hill, Beyond recommender systems: Helping people help each other, in HCI in the New Millennium, J.M. Carroll, Editor. 2001, Addison-Wesley: New York.
• 17: Steve Whittaker , Loren Terveen , Will Hill , Lynn Cherny, The dynamics of mass interaction, Proceedings of the 1998 ACM conference on Computer supported cooperative work, p.257-264, November 14-18, 1998, Seattle, Washington, United States [doi>10.1145/289444.289500]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces

General Terms:
Human Factors

Keywords:
collaborative filtering, computer-mediated communication, recommender systems