Crawling English-Japanese person-name transliterations from the web

ABSTRACT

Automatic compilation of lexicon is a dream of lexicon compilers as well as lexicon users. This paper proposes a system that crawls English-Japanese person-name transliterations from the Web, which works a back-end collector for automatic compilation of bilingual person-name lexicon. Our crawler collected 561K transliterations in five months. From them, an English-Japanese person-name lexicon with 406K entries has been compiled by an automatic post processing. This lexicon is much larger than other similar resources including English-Japanese lexicon of HeiNER obtained from Wikipedia.

REFERENCES

• 1: S. Kaide and S. Sato. A person-name classifier by using probability difference (in Japanese). In Proc. of NLP-09, 2009.
• 2: Masaaki Nagata , Teruka Saito , Kenji Suzuki, Using the web as a bilingual dictionary, Proceedings of the workshop on Data-driven methods in machine translation, p.1-8, July 07-07, 2001, Toulouse, France [doi>10.3115/1118037.1118050]
• 3: Y. Sakakibara and S. Sato. Automatic compilation of a bilingual person-name lexicon (in Japanese). In Proc. of NLP-07, pages 879--882, 2007.
• 4: W. Wentland, J. Knopp, C. Silberer, and M. Hartung. Building a multilingual lexical resource for named entity disambiguation, translation and transliteration. In Proc. of LREC-08, 2008.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Design, Experimentation

Keywords:
automatic lexicon compilation, mining transliteration pairs, person name