Mopping up: modeling wikipedia promotion decisions

ABSTRACT

This paper presents a model of the behavior of candidates for promotion to administrator status in Wikipedia. It uses a policy capture framework to highlight similarities and differences in the community's stated criteria for promotion decisions to those criteria actually correlated with promotion success. As promotions are determined by the consensus of dozens of voters with conflicting opinions and unwritten expectations, the results highlight the degree to which consensus is truly reached. The model is fast and easily computable on the fly, and thus could be applied as a self-evaluation tool for editors considering becoming administrators, as a dashboard for voters to view a nominee's relevant statistics, or as a tool to automatically search for likely future administrators. Implications for distributed consensus-building in online communities are discussed.

REFERENCES

• 1: Susan L. Bryant , Andrea Forte , Amy Bruckman, Becoming Wikipedian: transformation of participation in a collaborative online encyclopedia, Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work, November 06-09, 2005, Sanibel Island, Florida, USA [doi>10.1145/1099203.1099205]
• 2: Brian Butler , Elisabeth Joyce , Jacqueline Pike, Don't look now, but we've created a bureaucracy: the nature and roles of policies and rules in wikipedia, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357227]
• 3: Collier, B., Burke, M., Kittur, N. and Kraut, R. E., Retrospective versus Prospective Evidence for Promotion: The Case of Wikipedia. In 2008 annual meeting of the Academy of Management, (Anaheim, California, 2008), Academy of Management.
• 4: Dan Cosley , Dan Frankowski , Loren Terveen , John Riedl, SuggestBot: using intelligent task routing to help people find work in wikipedia, Proceedings of the 12th international conference on Intelligent user interfaces, January 28-31, 2007, Honolulu, Hawaii, USA [doi>10.1145/1216295.1216309]
• 5: Dawes, R. M. (1979). The robust beauty of improper linear models in decision making. American Psychologist, 34(7), 571--582.
• 6: Nicolas Ducheneaut, Socialization in an Open Source Software Community: A Socio-Technical Analysis, Computer Supported Cooperative Work, v.14 n.4, p.323-368, August 2005 [doi>10.1007/s10606-005-9000-1]
• 7: Andrea Forte , Amy Bruckman, Scaling Consensus: Increasing Decentralization in Wikipedia Governance, Proceedings of the Proceedings of the 41st Annual Hawaii International Conference on System Sciences, p.157, January 07-10, 2008 [doi>10.1109/HICSS.2008.383]
• 8: Giles, G. (2005). Internet Encyclopedias Go Head to Head. Nature, 438(7070), 900--901.
• 9: Hastie, R., & Dawes, R. M. (2001). Rational choice in an uncertain world: The psychology of judgment and decision making. Thousand Oaks, CA: Sage Publications.
• 10: Hobson, C. J., & Gibson, F. W. (1983). Policy Capturing as an Approach to Understanding and Improving Performance Appraisal: A Review of the Literature. The Academy of Management Review, 8(4), 640--649.
• 11: Hocking, R. (2003). Collinearity in multiple regression. Chapter 5 of Methods and Applications of Linear Models: Regression and the Analysis of Variance. Hoboken, NJ: Wiley-Interscience..
• 12: Karau, S., and Williams, K. Social loafing: A meta-analytic review and theoretical integration. Journal of Personality and Social Psychology, 65 (1993), 681--706.
• 13: Kittur, A., Chi, E., Pendleton, B., Suh, B., and Mytkowicz, T. Power of the few vs. wisdom of the crowd: Wikipedia and the rise of the bourgeoisie. Proc CHI 2007, ACM Press (2007).
• 14: Aniket Kittur , Bongwon Suh , Bryan A. Pendleton , Ed H. Chi, He says, she says: conflict and coordination in Wikipedia, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240698]
• 15: Locke, E. A. and Latham, G. P. Building a practically useful theory of goal setting and task motivation: A 35 year odyssey. American Psychologist, 57 (9). 705--717.
• 16: Meehl, P. E. Clinical vs. statistical prediction: A theoretical analysis and review of the literature. Menneapolis, University of Minnesota Press, 1954.
• 17: Ng, T. W. H., Eby, L. T., Sorensen, K. L., & Feldman, D. C. (2005). Predictors of objective and subjective career success: A meta-analysis. Personnel Psychology, 58(2), 367--409.
• 18: Reid Priedhorsky , Jilin Chen , Shyong (Tony) K. Lam , Katherine Panciera , Loren Terveen , John Riedl, Creating, destroying, and restoring value in wikipedia, Proceedings of the 2007 international ACM conference on Supporting group work, November 04-07, 2007, Sanibel Island, Florida, USA [doi>10.1145/1316624.1316663]
• 19: Stumpf, S. A., & London, M. (1981). Capturing rater policies in evaluating candidates for promotion. The Academy of Management Journal, 24(4), 752--766.
• 20: Fernanda B. Viégas , Martin Wattenberg , Kushal Dave, Studying cooperation and conflict between authors with history flow visualizations, Proceedings of the SIGCHI conference on Human factors in computing systems, p.575-582, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985765]
• 21: Fernanda B. Viegas , Martin Wattenberg , Jesse Kriss , Frank van Ham, Talk Before You Type: Coordination in Wikipedia, Proceedings of the 40th Annual Hawaii International Conference on System Sciences, p.78, January 03-06, 2007 [doi>10.1109/HICSS.2007.511]
• 22: Vroom, V., Porter, L., and Lawler, E. Expectancy Theories. in Miner, J. B. ed. Organizational Behavior 1: Essential Theories of Motivation and Leadership, ME Sharpe, Armonk NY, 2005, 94--113.
• 23: Zedeck, S., & Kafry, D. (1977). Capturing Rater Policies for Processing Evaluation Data. Organizational Behavior and Human Performance, 18(2), 269--294.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces
          Subjects: Collaborative computing

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces
          Subjects: Web-based interaction; Computer-supported cooperative work

General Terms:
Experimentation

Keywords:
administrators, collaboration, management, organizational behavior, policy capture, promotion, wikipedia