Translating unknown queries with web corpora for cross-language information retrieval

ABSTRACT

It is crucial for cross-language information retrieval (CLIR) systems to deal with the translation of unknown queries due to that real queries might be short. The purpose of this paper is to investigate the feasibility of exploiting the Web as the corpus source to translate unknown queries for CLIR. We propose an online translation approach to determine effective translations for unknown query terms via mining of bilingual search-result pages obtained from Web search engines. This approach can alleviate the problem of the lack of large bilingual corpora, translate many unknown query terms, provide flexible query specifications, and extract semantically-close translations to benefit CLIR tasks -- especially for cross-language Web search.

REFERENCES

• 1: Lee-Feng Chien, PAT-tree-based keyword extraction for Chinese information retrieval, Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval, p.50-58, July 27-31, 1997, Philadelphia, Pennsylvania, United States
• 2: Pascale Fung , Lo Yuen Yee, An IR approach for translating new words from nonparallel, comparable texts, Proceedings of the 17th international conference on Computational linguistics, August 10-14, 1998, Montreal, Quebec, Canada
• 3: William A. Gale , Kenneth W. Church, Identifying word correspondence in parallel texts, Proceedings of the workshop on Speech and Natural Language, p.152-157, February 19-22, 1991, Pacific Grove, California [doi>10.3115/112405.112428]
• 4: David A. Hull , Gregory Grefenstette, Querying across languages: a dictionary-based approach to multilingual information retrieval, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.49-57, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243212]
• 5: Adam Kilgarriff , Gregory Grefenstette, Introduction to the special issue on the web as corpus, Computational Linguistics, v.29 n.3, p.333-347, September 2003 [doi>10.1162/089120103322711569]
• 6: Julian Kupiec, An algorithm for finding noun phrase correspondences in bilingual corpora, Proceedings of the 31st annual meeting on Association for Computational Linguistics, p.17-22, June 22-26, 1993, Columbus, Ohio [doi>10.3115/981574.981577]
• 7: Kwok, K.L. NTCIR-2 Chinese, Cross Language Retrieval Experiments Using PIRCS. In Proc. of NTCIR Workshop Meeting, 2001.
• 8: Victor Lavrenko , Martin Choquette , W. Bruce Croft, Cross-lingual relevance models, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564408]
• 9: Wen-Hsiang Lu , Lee-Feng Chien , Hsi-Jian Lee, Anchor text mining for translation of Web queries: A transitive translation approach, ACM Transactions on Information Systems (TOIS), v.22 n.2, p.242-269, April 2004 [doi>10.1145/984321.984324]
• 10: I. Dan Melamed, Models of translational equivalence among words, Computational Linguistics, v.26 n.2, p.221-249, June 2000 [doi>10.1162/089120100561683]
• 11: Jian-Yun Nie , Michel Simard , Pierre Isabelle , Richard Durand, Cross-language information retrieval based on parallel texts and automatic mining of parallel texts from the Web, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.74-81, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312656]
• 12: Oard, D.W. and Diekema, A.R. Cross-Language Information Retrieval. In Annual Review of Information Science and Technology. American Society for Information Science, 1998.
• 13: Hsiao-Tieh Pu , Shui-Lung Chuang , Chyan Yang, Subject categorization of query terms for exploring Web users' search interests, Journal of the American Society for Information Science and Technology, v.53 n.8, p.617-630, August 2002 [doi>10.1002/asi.10071]
• 14: Reinhard Rapp, Automatic identification of word translations from unrelated English and German corpora, Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, p.519-526, June 20-26, 1999, College Park, Maryland [doi>10.3115/1034678.1034756]
• 15: Philip Resnik, Mining the Web for bilingual text, Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, p.527-534, June 20-26, 1999, College Park, Maryland [doi>10.3115/1034678.1034757]
• 16: Joaquim Ferreira da Silva , Gaël Dias , Sylvie Guilloré , José Gabriel Pereira Lopes, Using LocalMaxs Algorithm for the Extraction of Contiguous and Non-contiguous Multiword Lexical Units, Proceedings of the 9th Portuguese Conference on Artificial Intelligence: Progress in Artificial Intelligence, p.113-132, September 21-24, 1999
• 17: Silverstein, C., Henzinger, M., Marais, H., and Morics, M. Analysis of a Very Large AltaVista Query Log. Technical Report 1998-014, Digital Systems Research Center, 1998.
• 18: Frank Smadja , Kathleen R. McKeown , Vasileios Hatzivassiloglou, Translating collocations for bilingual lexicons: a statistical approach, Computational Linguistics, v.22 n.1, p.1-38, March 1996
• 19: Voorhees, E.M. and Harman, D.K. Overview of the sixth Text Retrieval Conference TREC-6. In Proc. of the 6th Text Retrieval Conference, 1998.
• 20: Jenq-Haur Wang , Jei-Wen Teng , Pu-Jen Cheng , Wen-Hsiang Lu , Lee-Feng Chien, Translating unknown cross-lingual queries in digital libraries using a web-based approach, Proceedings of the 4th ACM/IEEE-CS joint conference on Digital libraries, June 07-11, 2004, Tuscon, AZ, USA [doi>10.1145/996350.996378]
• 21: Jinxi Xu , Ralph Weischedel , Chanh Nguyen, Evaluating a probabilistic model for cross-lingual information retrieval, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.105-110, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383968]
• 22: Christopher C. Yang , Kar Wing Li, Automatic construction of English/Chinese parallel corpora, Journal of the American Society for Information Science and Technology, v.54 n.8, p.730-742, June 2003 [doi>10.1002/asi.10261]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Experimentation, Performance

Keywords:
cross-language information retrieval, cross-language web search, query translation