Coordination in collective intelligence: the role of team structure and task interdependence

ABSTRACT

The success of Wikipedia has demonstrated the power of peer production in knowledge building. However, unlike many other examples of collective intelligence, tasks in Wikipedia can be deeply interdependent and may incur high coordination costs among editors. Increasing the number of editors increases the resources available to the system, but it also raises the costs of coordination. This suggests that the dependencies of tasks in Wikipedia may determine whether they benefit from increasing the number of editors involved. Specifically, we hypothesize that adding editors may benefit low-coordination tasks but have negative consequences for tasks requiring a high degree of coordination. Furthermore, concentrating the work to reduce coordination dependencies should enable more efficient work by many editors. Analyses of both article ratings and article review comments provide support for both hypotheses. These results suggest ways to better harness the efforts of many editors in social collaborative systems involving high coordination tasks.

REFERENCES

• 1: Aiken, L.&West, S. Multiple regression: Testing and interpreting interactions. Sage, CA (1991).
• 2: Argote, L. Input uncertainty and organizational coordination in hospital emergency rooms. Administrative Science Quarterly 27 (1982), 420--434.
• 3: Atkinson, A.B. On the measurement of inequality. Journal of Economic Theory 2 (1970), 244--263.
• 4: Ronald M. Baecker , Dimitrios Nastos , Ilona R. Posner , Kelly L. Mawby, The user-centered iterative design of collaborative writing software, Proceedings of the INTERACT '93 and CHI '93 conference on Human factors in computing systems, p.399-405, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169312]
• 5: Benkler, Y. Coase's penguin, or Linux and the nature of the firm. Yale Law Journal 112 (2002), 367--445.
• 6: Frederick P. Brooks, Jr., The Mythical Man-Month: Essays on Softw, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1978
• 7: Cataldo, M., Wagstrom, P., Herbsleb, J.D.,&Carley, K. Identification of coordination requirements: Implications for the design of collaboration and awareness tools. ACM Press, New York, 2006.
• 8: Cohen, J., Cohen, P., West, S.G.,&Aiken, L.S. Applied multiple regression/correlation analysis for the behavioral sciences. Lawrence Erlbaum Associates, Mahwah, New Jersey (2003).
• 9: Dan Cosley , Dan Frankowski , Loren Terveen , John Riedl, SuggestBot: using intelligent task routing to help people find work in wikipedia, Proceedings of the 12th international conference on Intelligent user interfaces, January 28-31, 2007, Honolulu, Hawaii, USA [doi>10.1145/1216295.1216309]
• 10: Galton, F. Vox populi. Nature 75 (1907), 7.
• 11: Giles, G. Internet encyclopaedias go head to head. Nature 438 (2005), 900--901.
• 12: Jody Hoffer Gittell, Coordinating Mechanisms in Care Provider Groups: Relational Coordination as a Mediator and Input Uncertainty as a Moderator of Performance Effects, Management Science, v.48 n.11, p.1408-1426, November 2002 [doi>10.1287/mnsc.48.11.1408.268]
• 13: Hill, G.W. Group versus individual performance. Are N = 1 heads better than one? Psychological Bulletin 91, 3 (1982), 517--539.
• 14: Wikipedia Database Download. http://en.wikipedia.org/wiki/Wikipedia:Database_download
• 15: Wikipedia Featured Article Criteria. http://en.wikipedia.org/wiki/Wikipedia:Featured_article_criteria
• 16: Wikipedia Featured Article Statistics. http://en.wikipedia.org/wiki/Wikipedia:Featured_article_statistics
• 17: Aniket Kittur , Bongwon Suh , Bryan A. Pendleton , Ed H. Chi, He says, she says: conflict and coordination in Wikipedia, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240698]
• 18: Aniket Kittur , Ed H. Chi , Bongwon Suh, Crowdsourcing user studies with Mechanical Turk, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357127]
• 19: Kittur, A., Chi, E., Pendleton, B. A., Suh, B.,&Mytkowicz, T. Power of the few vs. wisdom of the crowd: Wikipedia and the rise of the bourgeoisie. Alt.CHI, 2007, San Jose, CA (2007).
• 20: Aniket Kittur , Robert E. Kraut, Harnessing the wisdom of crowds in wikipedia: quality through coordination, Proceedings of the ACM 2008 conference on Computer supported cooperative work, November 08-12, 2008, San Diego, CA, USA [doi>10.1145/1460563.1460572]
• 21: Kozlow, M.,&Bellamy, P. Experimental study on the impact of the 6+1 trait writing model on student achievement in writing. Association for Supervision and Curriculum Development Conference 2005, Orlando, FL (2005).
• 22: Thomas W. Malone , Kevin Crowston, The interdisciplinary study of coordination, ACM Computing Surveys (CSUR), v.26 n.1, p.87-119, March 1994 [doi>10.1145/174666.174668]
• 23: Audris Mockus , Roy T. Fielding , James D. Herbsleb, Two case studies of open source software development: Apache and Mozilla, ACM Transactions on Software Engineering and Methodology (TOSEM), v.11 n.3, p.309-346, July 2002 [doi>10.1145/567793.567795]
• 24: Northwest Regional Educational Laboratory. 6+1 Trait Writing: Scoring Continuum. (2001). Available online http://www.nwrel.org/assessment/pdfRubrics/6plus1traits.PDF
• 25: O'Reilly, T. (2005, Sep 30). What is Web 2.0? Design patterns and business models for the next generation of software. Retrieved Aug 15, 2008, from http://www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/whatis-web-20.html
• 26: Reid Priedhorsky , Jilin Chen , Shyong (Tony) K. Lam , Katherine Panciera , Loren Terveen , John Riedl, Creating, destroying, and restoring value in wikipedia, Proceedings of the 2007 international ACM conference on Supporting group work, November 04-07, 2007, Sanibel Island, Florida, USA [doi>10.1145/1316624.1316663]
• 27: Raymond, E. The cathedral and the bazaar. Knowledge, Technology, and Policy 12 (1999), 23 -- 49.
• 28: Rouse, W.B., Cannon-Bowers, J. A.,&Salas, E. The role of mental models in team performance in complex systems. IEEE Transactions on Systems, Man, and Cybernetics 22 (1992) 1296--1308.
• 29: Steiner, I. D. Group process and productivity. New York: Academic Press (1972).
• 30: Stewart, G. L. A meta-analytic review of relationships between team design features and team performance. Journal of Management 32, 1 (2006), 29.
• 31: Stvilia, B., Twidale, M. B., Smith, L. C.,&Gasser, L. Assessing information quality of a community-based encyclopedia. Proc. ICIQ 2005, MIT Press (2005), 442--454.
• 32: James Surowiecki, The Wisdom of Crowds, Anchor, 2005
• 33: Tushman, M. Work characteristics and subunit communication structure: A contingency analysis. Administrative Science Quarterly 24 (1979), 82--98.
• 34: Van de Ven, A., Delbecq, A.,&Koenig, R. Determinants of coordination modes within organizations. American Sociological Review 41 (1976), 322--338.
• 35: Wilkinson, D.M.,&Huberman, B.A. Assessing the value of cooperation in Wikipedia. First Monday 12, 4 (2007).

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces

Additional Classification:
  K. Computing Milieux
  K.4 COMPUTERS AND SOCIETY
      K.4.3 Organizational Impacts

General Terms:
Human Factors

Keywords:
Wiki, Wikipedia, collective intelligence, coordination, social collaboration, social computing