Information arbitrage across multi-lingual Wikipedia

ABSTRACT

The rapid globalization of Wikipedia is generating a parallel, multi-lingual corpus of unprecedented scale. Pages for the same topic in many different languages emerge both as a result of manual translation and independent development. Unfortunately, these pages may appear at different times, vary in size, scope, and quality. Furthermore, differential growth rates cause the conceptual mapping between articles in different languages to be both complex and dynamic. These disparities provide the opportunity for a powerful form of information arbitrage--leveraging articles in one or more languages to improve the content in another. Analyzing four large language domains (English, Spanish, French, and German), we present Ziggurat, an automated system for aligning Wikipedia infoboxes, creating new infoboxes as necessary, filling in missing information, and detecting discrepancies between parallel pages. Our method uses self-supervised learning and our experiments demonstrate the method's feasibility, even in the absence of dictionaries.

REFERENCES

• 1: Sisay Fissaha Adafre , Maarten de Rijke, Discovering missing links in Wikipedia, Proceedings of the 3rd international workshop on Link discovery, p.90-97, August 21-25, 2005, Chicago, Illinois [doi>10.1145/1134271.1134284]
• 2: Adafre, S. F., and M. de Rijke, "Finding Similar Sentences across Multiple Languages in Wikipedia," EACL '06, Trento, Italy, April 2006.
• 3: DBpedia, www.dbpedia.org, last retrieved Aug. 10, 2008.
• 4: Etzioni, O., K. Reiter, S. Soderland, and M. Sammer, "Lexical translation with application to image searching on the web." MT Summit XI, Copenhagen, Denmark, September, 2007.
• 5: Ferrández, S., A. Toral, Ó. Ferrández, A. Ferrández, and R. Muñoz, "Applying Wikipedia's Multilingual Knowledge to Cross--Lingual Question Answering," Lecture Notes in Computer Science, vol. 4592, Springer, 2007.
• 6: Friedman, J., T. Hastie, and R. Tibshirani, "Additive logistic regression: A statistical view of boosting." Annals of Statistics, 28(20), 337--407, 2000
• 7: Kawaba, M., H. Nakasaki, T. Utsuro, and T. Fukuhara, "Cross-Lingual Blog Analysis based on Multilingual Blog Distillation from Multilingual Wikipedia Entries," ICWSM'08, Seattle, WA, March 2008.
• 8: Kinzler, D., "Automatischer Aufbau eines multilingualen Thesaurus durch Extraktion semantischer und lexikalischer Relationen aus der Wikipedia" Thesis, 2008.
• 9: Andrew McCallum , Ronald Rosenfeld , Tom M. Mitchell , Andrew Y. Ng, Improving Text Classification by Shrinkage in a Hierarchy of Classes, Proceedings of the Fifteenth International Conference on Machine Learning, p.359-367, July 24-27, 1998
• 10: Erhard Rahm , Philip A. Bernstein, A survey of approaches to automatic schema matching, The VLDB Journal — The International Journal on Very Large Data Bases, v.10 n.4, p.334-350, December 2001 [doi>10.1007/s007780100057]
• 11: Richman, A. E., and P Schone, "Mining Wiki Resources for Multilingual Named Entity Recognition," ACL'08, Columbus, Ohio, June 2008.
• 12: Sorg, P., and P. Cimiano, "Enriching the Crosslingual Link Structure of Wikipedia - A Classification-Based Approach," AAAI'08 Wikipedia and Artificial Intelligence Workshop, Chicago, IL, July 2008.
• 13: Michael Toomim , Andrew Begel , Susan L. Graham, Managing Duplicated Code with Linked Editing, Proceedings of the 2004 IEEE Symposium on Visual Languages - Human Centric Computing, p.173-180, September 26-29, 2004 [doi>10.1109/VLHCC.2004.35]
• 14: Voss, J, "Measuring Wikipedia." 10^th International Conference of the International Society for Scientometrics and Informetrics, Stockholm, Sweden. 2005.
• 15: Weld, Daniel S., F. Wu, E. Adar, S. Amershi, J. Fogarty, R. Hoffman, K. Patel, and M. Skinner, "Intelligence in Wikipedia," AAAI'08, Chicago, IL, July 2008.
• 16: Wentald, W., J. Knopp, C. Silberer, and M. Hartung, "Building a Multilingual Lexical Resource for Named Entity Disambiguation, Translation and Transliteration," LREC '08, Marrakech, Morocco, May 2008.
• 17: "Wikipedia: MultiLingual Statistics," Aug. 10, 2008 en.wikipedia.org/wiki/Wikipedia:Multilingual_statistics,
• 18: Fei Wu , Raphael Hoffmann , Daniel S. Weld, Information extraction from Wikipedia: moving down the long tail, Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, August 24-27, 2008, Las Vegas, Nevada, USA [doi>10.1145/1401890.1401978]
• 19: Fei Wu , Daniel S. Weld, Automatically refining the wikipedia infobox ontology, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367583]
• 20: Fei Wu , Daniel S. Weld, Autonomously semantifying wikipedia, Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, November 06-10, 2007, Lisbon, Portugal [doi>10.1145/1321440.1321449]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.m Miscellaneous

General Terms:
Algorithms, Experimentation

Keywords:
Wikipedia, information arbitrage, multi-lingual, translation