SuggestBot: using intelligent task routing to help people find work in wikipedia

ABSTRACT

Member-maintained communities ask their users to perform tasks the community needs. From Slashdot, to IMDb, to Wikipedia, groups with diverse interests create community-maintained artifacts of lasting value (CALV) that support the group's main purpose and provide value to others. Said communities don't help members find work to do, or do so without regard to individual preferences, such as Slashdot assigning meta-moderation randomly. Yet social science theory suggests that reducing the cost and increasing the personal value of contribution would motivate members to participate more.We present SuggestBot, software that performs intelligent task routing (matching people with tasks) in Wikipedia. SuggestBot uses broadly applicable strategies of text analysis, collaborative filtering, and hyperlink following to recommend tasks. SuggestBot's intelligent task routing increases the number of edits by roughly four times compared to suggesting random articles. Our contributions are: 1) demonstrating the value of intelligent task routing in a real deployment; 2) showing how to do intelligent task routing; and 3) sharing our experience of deploying a tool in Wikipedia, which offered both challenges and opportunities for research.

REFERENCES

• 1: Gerard Beenen , Kimberly Ling , Xiaoqing Wang , Klarissa Chang , Dan Frankowski , Paul Resnick , Robert E. Kraut, Using social psychology to motivate contributions to online communities, Proceedings of the 2004 ACM conference on Computer supported cooperative work, November 06-10, 2004, Chicago, Illinois, USA [doi>10.1145/1031607.1031642]
• 2: Susan L. Bryant , Andrea Forte , Amy Bruckman, Becoming Wikipedian: transformation of participation in a collaborative online encyclopedia, Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work, November 06-09, 2005, Sanibel Island, Florida, USA [doi>10.1145/1099203.1099205]
• 3: Robin Burke, Hybrid Recommender Systems: Survey and Experiments, User Modeling and User-Adapted Interaction, v.12 n.4, p.331-370, November 2002 [doi>10.1023/A:1021240730564]
• 4: A. Ciffolilli. Phantom authority, self-selective recruitment and retention of members in virtual communities: The case of wikipedia. First Monday, 8(12), 2003.
• 5: T. Connolly and B. Thorn. Discretionary databases: Theory, data, and implications. Organizations and Communication Technology, pages 219--233, 1990.
• 6: Dan Cosley , Dan Frankowski , Sara Kiesler , Loren Terveen , John Riedl, How oversight improves member-maintained communities, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1054975]
• 7: Dan Cosley , Dan Frankowski , Loren Terveen , John Riedl, Using intelligent task routing and contribution review to help communities build artifacts of lasting value, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124928]
• 8: D. Cosley, S. Lawrence, and D. M. Pennock. REFEREE: An open framework for practical testing of recommender systems using researchindex. In VLDB'02, Hong Kong, August 20--23 2002.
• 9: Dan Cosley , Pamela Ludford , Loren Terveen, Studying the effect of similarity in online task-focused interactions, Proceedings of the 2003 international ACM SIGGROUP conference on Supporting group work, November 09-12, 2003, Sanibel Island, Florida, USA [doi>10.1145/958160.958212]
• 10: William Emigh , Susan C. Herring, Collaborative Authoring on the Web: A Genre Analysis of Online Encyclopedias, Proceedings of the Proceedings of the 38th Annual Hawaii International Conference on System Sciences (HICSS'05) - Track 4, p.99.1, January 03-06, 2005 [doi>10.1109/HICSS.2005.149]
• 11: R. Hardin. Collective Action. Johns Hopkins, Baltimore, 1982.
• 12: S. G. Harkins. Social loafing and social facilitation. Journal of Experimental Social Psych., 23:1--18, 1987.
• 13: F. M. Harper, X. Li, Y. Chen, and J. A. Konstan. An economic model of user rating in an online recommender system. In User Modeling, page 307, Edinburgh, Scotland, 2005. Springer Berlin.
• 14: Jonathan L. Herlocker , Joseph A. Konstan , Al Borchers , John Riedl, An algorithmic framework for performing collaborative filtering, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.230-237, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312682]
• 15: Jonathan L. Herlocker , Joseph A. Konstan , Loren G. Terveen , John T. Riedl, Evaluating collaborative filtering recommender systems, ACM Transactions on Information Systems (TOIS), v.22 n.1, p.5-53, January 2004 [doi>10.1145/963770.963772]
• 16: William C. Hill , James D. Hollan , Dave Wroblewski , Tim McCandless, Edit wear and read wear, Proceedings of the SIGCHI conference on Human factors in computing systems, p.3-9, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142751]
• 17: Christoph Höscher , Gerhard Strube, Web search behavior of Internet experts and newbies, Proceedings of the 9th international World Wide Web conference on Computer networks : the international journal of computer and telecommunications netowrking, p.337-346, June 2000, Amsterdam, The Netherlands
• 18: E. Joyce and R. E. Kraut. Predicting continued participation in newsgroups. JCMC, 11(3), 2006.
• 19: N. L. Kerr. Motivation losses in small groups: a social dilemma analysis. J. Pers. Soc. Psych., 45:819--828, 1983.
• 20: K. Lakhani and E. von Hippel. How open source software works: "free" user-to-user assistance. Research Policy, 32:923--943, 2002.
• 21: Cliff Lampe , Paul Resnick, Slash(dot) and burn: distributed moderation in a large online conversation space, Proceedings of the SIGCHI conference on Human factors in computing systems, p.543-550, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985761]
• 22: B. Latané, K. Williams, and S. Harkins. Many hands make light the work: The causes and consequences of social loafing. J. Pers. Soc. Psych., 37:822--832, 1979.
• 23: J. Lave and E. Wenger. Situated learning: Legitimate peripheral participation. Cambridge University, New York, NY, 1991.
• 24: J. O. Ledyard. Public Goods: A Survey of Experimental Research, pages 111--194. The Handbook of Experimental Research. Princeton University Press, Princeton, NJ, 1995.
• 25: A. Lih. Wikipedia as participatory journalism: Reliable sources? metrics for evaluating collaborative media as a news resource. In 5th International Symposium on Online Journalism, Austin, TX, 2004.
• 26: K. Ling, G. Beenen, P. Ludford, X. Wang, K. Chang, X. Li, D. Cosley, D. Frankowski, and L. Terveen. Using social psychology to motivate contributions to online communities. JCMC, 10(4), 2005.
• 27: Pamela J. Ludford , Dan Cosley , Dan Frankowski , Loren Terveen, Think different: increasing online community participation using uniqueness and group dissimilarity, Proceedings of the SIGCHI conference on Human factors in computing systems, p.631-638, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985772]
• 28: Sean M. McNee , Istvan Albert , Dan Cosley , Prateep Gopalkrishnan , Shyong K. Lam , Al Mamunur Rashid , Joseph A. Konstan , John Riedl, On the recommending of citations for research papers, Proceedings of the 2002 ACM conference on Computer supported cooperative work, November 16-20, 2002, New Orleans, Louisiana, USA [doi>10.1145/587078.587096]
• 29: S. M. McNee, S. K. Lam, J. A. Konstan, and J. Riedl. Interfaces for eliciting new user preferences in recommender systems. In User Modeling, pages 178--187, Johnstown, PA, USA, 2003. Springer Verlag.
• 30: Masahiro Morita , Yoichi Shinoda, Information filtering based on user behavior analysis and best match text retrieval, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.272-281, July 03-06, 1994, Dublin, Ireland
• 31: Bonnie A. Nardi, Context and consciousness: activity theory and human-computer interaction, Massachusetts Institute of Technology, Cambridge, MA, 1995
• 32: Gregory B. Newby , Charles Franks, Distributed proofreading, Proceedings of the 3rd ACM/IEEE-CS joint conference on Digital libraries, May 27-31, 2003, Houston, Texas
• 33: John O'Donovan , Barry Smyth, Is trust robust?: an analysis of trust-based recommendation, Proceedings of the 11th international conference on Intelligent user interfaces, January 29-February 01, 2006, Sydney, Australia [doi>10.1145/1111449.1111476]
• 34: Al M. Rashid , Kimberly Ling , Regina D. Tassone , Paul Resnick , Robert Kraut , John Riedl, Motivating participation by displaying the value of contribution, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124915]
• 35: Paul Resnick , Neophytos Iacovou , Mitesh Suchak , Peter Bergstrom , John Riedl, GroupLens: an open architecture for collaborative filtering of netnews, Proceedings of the 1994 ACM conference on Computer supported cooperative work, p.175-186, October 22-26, 1994, Chapel Hill, North Carolina, United States [doi>10.1145/192844.192905]
• 36: Gerard Salton , Michael J. McGill, Introduction to Modern Information Retrieval, McGraw-Hill, Inc., New York, NY, 1986
• 37: Badrul Sarwar , George Karypis , Joseph Konstan , John Reidl, Item-based collaborative filtering recommendation algorithms, Proceedings of the 10th international conference on World Wide Web, p.285-295, May 01-05, 2001, Hong Kong, Hong Kong [doi>10.1145/371920.372071]
• 38: Craig Silverstein , Hannes Marais , Monika Henzinger , Michael Moricz, Analysis of a very large web search engine query log, ACM SIGIR Forum, v.33 n.1, p.6-12, Fall 1999 [doi>10.1145/331403.331405]
• 39: B. Stvilia, M. Twidale, L. Smith, and L. Gasser. Assessing information quality of a community-based encyclopedia. In Proc. International Conference on Information Quality, pages 442--454, 2005.
• 40: Fernanda B. Viégas , Martin Wattenberg , Kushal Dave, Studying cooperation and conflict between authors with history flow visualizations, Proceedings of the SIGCHI conference on Human factors in computing systems, p.575-582, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985765]
• 41: J. Voss. Measuring wikipedia. In Proc. International Conference of the International Society for Scientometrics and Informetrics, Stockholm, 2005.
• 42: Cai-Nicolas Ziegler , Georg Lausen , Lars Schmidt-Thieme, Taxonomy-driven computation of product recommendations, Proceedings of the thirteenth ACM international conference on Information and knowledge management, November 08-13, 2004, Washington, D.C., USA [doi>10.1145/1031171.1031252]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces
          Subjects: Collaborative computing

General Terms:
Algorithms, Design, Economics, Theory

Keywords:
Wikipedia, intelligent task routing, member-maintained communities, online communities, recommender systems