Generating diverse katakana variants based on phonemic mapping

ABSTRACT

In Japanese, it is quite common for the same word to be written in several different ways. This is especially true for katakana words which are typically used for transliterating foreign languages. This ambiguity becomes critical for automatic processing such as information retrieval (IR). To tackle this problem, we propose a simple but effective approach to generating katakana variants by considering phonemic representation of the original language for a given word. The proposed approach is evaluated through an assessment of the variants it generates. Also, the impact of the generated variants on IR is studied in comparison to an existing approach using katakana rewriting rules.

REFERENCES

• 1: Kevin Knight , Jonathan Graehl, Machine transliteration, Computational Linguistics, v.24 n.4, p.599-612, December 1998
• 2: C. Kubomura and H. Kameda. Information retrieval system with abilities of processing katakana allographs. IEICE Trans. Inf. & Syst., J86-D-II(3):418--428, 2003. (In Japanese)
• 3: M. Shishibori, K. Tsuda, and J. Aoe. A method for generation and normalization of katakana variant notations. IEICE Trans. Info. & Syst., J77-D-II(2):380--387, 1994. (In Japanese)

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Query formulation

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Language models

General Terms:
Algorithms, Experimentation, Languages

Keywords:
information retrieval, katakana variants, transliteration