Using the web for automated translation extraction in cross-language information retrieval

ABSTRACT

There have been significant advances in Cross-Language Information Retrieval (CLIR) in recent years. One of the major remaining reasons that CLIR does not perform as well as monolingual retrieval is the presence of out of vocabulary (OOV) terms. Previous work has either relied on manual intervention or has only been partially successful in solving this problem. We use a method that extends earlier work in this area by augmenting this with statistical analysis, and corpus-based translation disambiguation to dynamically discover translations of OOV terms. The method can be applied to both Chinese-English and English-Chinese CLIR, correctly extracting translations of OOV terms from the Web automatically, and thus is a significant improvement on earlier work.

REFERENCES

• 1: Nasreen AbdulJaleel , Leah S. Larkey, Statistical transliteration for english-arabic cross language information retrieval, Proceedings of the twelfth international conference on Information and knowledge management, November 03-08, 2003, New Orleans, LA, USA [doi>10.1145/956863.956890]
• 2: Lisa Ballesteros , W. Bruce Croft, Dictionary Methods for Cross-Lingual Information Retrieval, Proceedings of the 7th International Conference on Database and Expert Systems Applications, p.791-801, September 09-13, 1996
• 3: Lisa Ballesteros , W. Bruce Croft, Resolving ambiguity for cross-language retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.64-71, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290958]
• 4: A. Chen and F. Gey. Experiments on Cross-language and Patent Retrieval at NTCIR-3 Workshop. In Proceedings of the 3rd NTCIR Workshop Japan, 2003.
• 5: Aitao Chen , Hailing Jiang , Fredric Gey, Combining multiple sources for short query translation in Chinese-English cross-language information retrieval, Proceedings of the fifth international workshop on on Information retrieval with Asian languages, p.17-23, September 30-October 01, 2000, Hong Kong, China [doi>10.1145/355214.355217]
• 6: Marcello Federico , Nicola Bertoldi, Statistical cross-language information retrieval using n-best query translations, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564407]
• 7: Jianfeng Gao , Ming Zhou , Jian-Yun Nie , Hongzhao He , Weijun Chen, Resolving query translation ambiguity using a decaying co-occurrence model and syntactic dependence relations, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564409]
• 8: Wessel Kraaij, TNO at CLEF-2001: Comparing Translation Resources, Revised Papers from the Second Workshop of the Cross-Language Evaluation Forum on Evaluation of Cross-Language Information Retrieval Systems, p.78-93, September 03-04, 2001
• 9: S. Kuai, Q. Lu, and L. Yang, editors. A New English-Chinese Dictionary Joint Publishing Company, HongKong, 1st edition, 1984.
• 10: K. L. Kwok, Exploiting a Chinese-English bilingual wordlist for English-Chinese cross language information retrieval, Proceedings of the fifth international workshop on on Information retrieval with Asian languages, p.173-179, September 30-October 01, 2000, Hong Kong, China [doi>10.1145/355214.355239]
• 11: H. Meng, B. Chen, S. Khudanpur, G. Levow, W. Lo, D. Oard, P. Schone, K. Tang, H. Wang, and J. Wang. Mandarin-English Information (MEI): investigating translingual speech retrieval. In Computer Speech and Language 2003.
• 12: Mirna Adriani, Using Statistical Term Similarity for Sense Disambiguationin Cross-Language Information Retrieval, Information Retrieval, v.2 n.1, p.71-82, February 2000 [doi>10.1023/A:1009989801965]
• 13: M. F. Porter. An Algorithm for Suffix Stripping. Program (Automated Library and Information Systems) 14(3):130--137, 1980.
• 14: Ian H. Witten , Alistair Moffat , Timothy C. Bell, Managing gigabytes (2nd ed.): compressing and indexing documents and images, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1999
• 15: Y. Zhang and P. Vinese. Improved Cross-Language Information Retrieval via Disambiguation and Vocabulary Discovery. In Proceedings of the 8th Australasian Document Computing Symposium pages 3--7,2003.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Languages

Keywords:
OOV problem, cross-language IR, query translation