Amplifying community content creation with mixed initiative information extraction

ABSTRACT

Although existing work has explored both information extraction and community content creation, most research has focused on them in isolation. In contrast, we see the greatest leverage in the synergistic pairing of these methods as two interlocking feedback cycles. This paper explores the potential synergy promised if these cycles can be made to accelerate each other by exploiting the same edits to advance both community content creation and learning-based information extraction. We examine our proposed synergy in the context of Wikipedia infoboxes and the Kylin information extraction system. After developing and refining a set of interfaces to present the verification of Kylin extractions as a non primary task in the context of Wikipedia articles, we develop an innovative use of Web search advertising services to study people engaged in some other primary task. We demonstrate our proposed synergy by analyzing our deployment from two complementary perspectives: (1) we show we accelerate community content creation by using Kylin's information extraction to significantly increase the likelihood that a person visiting a Wikipedia article as a part of some other primary task will spontaneously choose to help improve the article's infobox, and (2) we show we accelerate information extraction by using contributions collected from people interacting with our designs to significantly improve Kylin's extraction performance.

REFERENCES

• 1: Susan L. Bryant , Andrea Forte , Amy Bruckman, Becoming Wikipedian: transformation of participation in a collaborative online encyclopedia, Proceedings of the 2005 international ACM SIGGROUP conference on Supporting group work, November 06-09, 2005, Sanibel Island, Florida, USA [doi>10.1145/1099203.1099205]
• 2: Dan Cosley , Dan Frankowski , Loren Terveen , John Riedl, SuggestBot: using intelligent task routing to help people find work in wikipedia, Proceedings of the 12th international conference on Intelligent user interfaces, January 28-31, 2007, Honolulu, Hawaii, USA [doi>10.1145/1216295.1216309]
• 3: Aron Culotta , Trausti Kristjansson , Andrew McCallum , Paul Viola, Corrective feedback and persistent learning for information extraction, Artificial Intelligence, v.170 n.14, p.1101-1122, October 2006 [doi>10.1016/j.artint.2006.08.001]
• 4: Pedro DeRose , Xiaoyong Chai , Byron J. Gao , Warren Shen , AnHai Doan , Philip Bohannon , Xiaojin Zhu, Building Community Wikipedias: A Machine-Human Partnership Approach, Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, p.646-655, April 07-12, 2008 [doi>10.1109/ICDE.2008.4497473]
• 5: C. Lee Giles , Kurt D. Bollacker , Steve Lawrence, CiteSeer: an automatic citation indexing system, Proceedings of the third ACM conference on Digital libraries, p.89-98, June 23-26, 1998, Pittsburgh, Pennsylvania, United States [doi>10.1145/276675.276685]
• 6: Jonathan Grudin, Groupware and social dynamics: eight challenges for developers, Communications of the ACM, v.37 n.1, p.92-105, Jan. 1994 [doi>10.1145/175222.175230]
• 7: Raphael Hoffmann , James Fogarty , Daniel S. Weld, Assieme: finding and leveraging implicit references in a web search interface for programmers, Proceedings of the 20th annual ACM symposium on User interface software and technology, October 07-10, 2007, Newport, Rhode Island, USA [doi>10.1145/1294211.1294216]
• 8: Eric Horvitz, Principles of mixed-initiative user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.159-166, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303030]
• 9: David F. Huynh , Robert C. Miller , David R. Karger, Enabling web browsers to augment web sites' filtering and sorting functionalities, Proceedings of the 19th annual ACM symposium on User interface software and technology, October 15-18, 2006, Montreux, Switzerland [doi>10.1145/1166253.1166274]
• 10: Stacey Kuznetsov, Motivations of contributors to Wikipedia, ACM SIGCAS Computers and Society, v.36 n.2, p.1-es, June 2006 [doi>10.1145/1215942.1215943]
• 11: Jennifer Mankoff , Scott E. Hudson , Gregory D. Abowd, Interaction techniques for ambiguity resolution in recognition-based interfaces, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.11-20, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354407]
• 12: Robert McCann , Warren Shen , AnHai Doan, Matching Schemas in Online Communities: A Web 2.0 Approach, Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, p.110-119, April 07-12, 2008 [doi>10.1109/ICDE.2008.4497419]
• 13: Daniel McFarlane, Comparison of four primary methods for coordinating the interruption of people in human-computer interaction, Human-Computer Interaction, v.17 n.1, p.63-139, March 2002 [doi>10.1207/S15327051HCI1701_2]
• 14: MediaWiki. http://www.mediawiki.org/.
• 15: Reid Priedhorsky , Jilin Chen , Shyong (Tony) K. Lam , Katherine Panciera , Loren Terveen , John Riedl, Creating, destroying, and restoring value in wikipedia, Proceedings of the 2007 international ACM conference on Supporting group work, November 04-07, 2007, Sanibel Island, Florida, USA [doi>10.1145/1316624.1316663]
• 16: Michael Shilman , Desney S. Tan , Patrice Simard, CueTIP: a mixed-initiative interface for correcting handwriting errors, Proceedings of the 19th annual ACM symposium on User interface software and technology, October 15-18, 2006, Montreux, Switzerland [doi>10.1145/1166253.1166304]
• 17: Luis von Ahn , Laura Dabbish, Labeling images with a computer game, Proceedings of the SIGCHI conference on Human factors in computing systems, p.319-326, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985733]
• 18: Luis von Ahn , Laura Dabbish, Designing games with a purpose, Communications of the ACM, v.51 n.8, August 2008 [doi>10.1145/1378704.1378719]
• 19: Voss, J. (2005). Measuring Wikipedia. International Conference of the International Society for Scientometrics and Informetrics (ISSI 2005), 221--231.
• 20: Wikipedia: AutoWikiBrowser. http://en.wikipedia.org/wiki/Wikipedia:AutoWikiBrowser.
• 21: Wikipedia: Be Bold. http://en.wikipedia.org/wiki/Wikipedia:Be_Bold.
• 22: Wikipedia: Bot Policy. http://en.wikipedia.org/wiki/Wikipedia:Bots.
• 23: Wikipedia: Cleanup Tags. http://en.wikipedia.org/wiki/Wikipedia:Template_messages/Cleanup.
• 24: Fei Wu , Raphael Hoffmann , Daniel S. Weld, Information extraction from Wikipedia: moving down the long tail, Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, August 24-27, 2008, Las Vegas, Nevada, USA [doi>10.1145/1401890.1401978]
• 25: Fei Wu , Daniel S. Weld, Autonomously semantifying wikipedia, Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, November 06-10, 2007, Lisbon, Portugal [doi>10.1145/1321440.1321449]
• 26: Ka-Ping Yee , Kirsten Swearingen , Kevin Li , Marti Hearst, Faceted metadata for image search and browsing, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642681]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems

General Terms:
Design

Keywords:
community content creation, information extraction, mixed-initiative interfaces