QuWi: quality control in Wikipedia

ABSTRACT

We propose and evaluate QuWi (Quality in Wikipedia), a framework for quality control in Wikipedia. We build upon a previous proposal by Mizzaro [11], who proposed a method for substituting and/or complementing peer review in scholarly publishing. Since articles in Wikipedia are never finished, and their authors change continuously, we define a modified algorithm that takes into account the different domain, with particular attention to the fact that authors contribute identifiable pieces of information that can be further modified by other authors.

The algorithm assigns quality scores to articles and contributors. The scores assigned to articles can be used, e.g., to let the reader understand how reliable are the articles he or she is looking at, or to help contributors in identifying low quality articles to be enhanced. The scores assigned to users measure the average quality of their contributions to Wikipedia and can be used, e.g., for conflict resolution policies based on the quality of involved users.

Our proposed algorithm is experimentally evaluated by analyzing the obtained quality scores on articles for deletion and featured articles, also on six temporal Wikipedia snapshots. Preliminary results demonstrate that the proposed algorithm seems to appropriately identify high and low quality articles, and that high quality authors produce more long-lived contributions than low quality authors.

REFERENCES

• 1: B.Thomas Adler, Jason Benterou, Krishnendu Chatterjee, Luca De Alfaro, Ian Pye, and Vishwanath Raman. Assigning Trust to Wikipedia Content. Technical Report UCSC-CRL-07-09, School of Engineering, University of California, Santa Cruz, CA, USA, November 2007.
• 2: B. Thomas Adler , Luca de Alfaro, A content-driven reputation system for the wikipedia, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242608]
• 3: Denise Anthony, Sean W. Smith, and Tim Williamson. The Quality of Open Source Production: Zealots and Good Samaritans in the Case of Wikipedia. Technical Report TR2007-606, Dartmouth College, Computer Science, Hanover, NH, September 2007.
• 4: Francesco Bellomi and Roberto Bonato. Network Analysis for Wikipedia. Proceedings of Wikimania, 2005.
• 5: Sergey Brin , Lawrence Page, The anatomy of a large-scale hypertextual Web search engine, Computer Networks and ISDN Systems, v.30 n.1-7, p.107-117, April 1, 1998
• 6: William Emigh , Susan C. Herring, Collaborative Authoring on the Web: A Genre Analysis of Online Encyclopedias, Proceedings of the Proceedings of the 38th Annual Hawaii International Conference on System Sciences (HICSS'05) - Track 4, p.99.1, January 03-06, 2005 [doi>10.1109/HICSS.2005.149]
• 7: Jim Giles. Special report: Internet encyclopedias go head to head. Nature, 438(15):900--901, 2005.
• 8: Jon M. Kleinberg, Authoritative sources in a hyperlinked environment, Journal of the ACM (JACM), v.46 n.5, p.604-632, Sept. 1999 [doi>10.1145/324133.324140]
• 9: Andrew Lih. Wikipedia as participatory journalism: Reliable source? metrics for evaluating collaborative media as a news resource. In 5th International Symposium on Online Journalism, 2004.
• 10: Meiqun Hu , Ee-Peng Lim , Aixin Sun , Hady Wirawan Lauw , Ba-Quy Vuong, Measuring article quality in wikipedia: models and evaluation, Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, November 06-10, 2007, Lisbon, Portugal [doi>10.1145/1321440.1321476]
• 11: Stefano Mizzaro. Quality control in scholarly publishing: A new proposal. Journal of the American Society for Information Science and Technology, 54(11):989--1005, 2003.
• 12: Besiki Stvilia , Les Gasser , Michael B. Twidale , Linda C. Smith, A framework for information quality assessment, Journal of the American Society for Information Science and Technology, v.58 n.12, p.1720-1733, October 2007 [doi>10.1002/asi.v58:12]
• 13: Besiki Stvilia , Michael B. Twidale , Linda C. Smith , Les Gasser, Information quality work organization in wikipedia, Journal of the American Society for Information Science and Technology, v.59 n.6, p.983-1001, April 2008 [doi>10.1002/asi.v59:6]
• 14: Fernanda B. Viegas, Martin Wattenberg, and Matthew M. McKeon. The hidden order of wikipedia. Proceedings of HCII, 2007.
• 15: Dennis M. Wilkinson and Bernardo A. Huberman. Assessing the Value of Coooperation in Wikipedia. First Monday 12(4) April, 2007.
• 16: Honglei Zeng , Maher A. Alhossaini , Li Ding , Richard Fikes , Deborah L. McGuinness, Computing trust from revision history, Proceedings of the 2006 International Conference on Privacy, Security and Trust: Bridge the Gap Between PST Technologies and Business Services, October 30-November 01, 2006, Markham, Ontario, Canada [doi>10.1145/1501434.1501445]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Measurement

Keywords:
QuWi, quality control, reputation, wikipedia