Cross-lingual relevance models

ABSTRACT

We propose a formal model of Cross-Language Information Retrieval that does not rely on either query translation or document translation. Our approach leverages recent advances in language modeling to directly estimate an accurate topic model in the target language, starting with a query in the source language. The model integrates popular techniques of disambiguation and query expansion in a unified formal framework. We describe how the topic model can be estimated with either a parallel corpus or a dictionary. We test the framework by constructing Chinese topic models from English queries and using them in the CLIR task of TREC9. The model achieves performance around 95% of the strong mono-lingual baseline in terms of average precision. In initial precision, our model outperforms the mono-lingual baseline by 20%. The main contribution of this work is the unified formal model which integrates techniques that are essential for effective Cross-Language Retrieval.

REFERENCES

• 1: J. Allan, J. Callan, W. B. Croft, L. A. Ballesteros, D. Byrd, R. Swan, and J. Xu. INQUERY does battle with TREC-6. In E. M. Voorhees and D. K. Harman, editors, Proceedings of the Sixth Text REtrieval Conference (TREC-6), pages 169--206, Gaithersburg, MD, November 1997. National Institute of Standards and Technology (NIST) and Defense Advanced Research Projects Agency (DARPA), Department of Commerce, National Institute of Standards and Technology.
• 2: Lisa Ballesteros , W. Bruce Croft, Phrasal translation and query expansion techniques for cross-language information retrieval, Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval, p.84-91, July 27-31, 1997, Philadelphia, Pennsylvania, United States
• 3: Lisa Ballesteros , W. Bruce Croft, Resolving ambiguity for cross-language retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.64-71, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290958]
• 4: Adam Berger , John Lafferty, Information retrieval as statistical translation, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.222-229, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312681]
• 5: Peter F. Brown , John Cocke , Stephen A. Della Pietra , Vincent J. Della Pietra , Fredrick Jelinek , John D. Lafferty , Robert L. Mercer , Paul S. Roossin, A statistical approach to machine translation, Computational Linguistics, v.16 n.2, p.79-85, June 1990
• 6: W. B. Croft, D. J. Harper, D. H. Kraft, and J. Zobel, editors. Proceedings of the Twenty-Fourth Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, New Orleans, LA, September 2001. ACM Press.
• 7: J. Gao, J.-Y. Nie, J. Zhang, E. Xun, Y. Su, M. Zhou, and C. Huang. TREC-9 CLIR experiments at MSRCN. In Voorhees and Harman {17}, pages 343--354.
• 8: Tim Gollins , Mark Sanderson, Improving cross language retrieval with triangulated translation, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.90-95, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383965]
• 9: Djoerd Hiemstra , Franciska de Jong, Disambiguation Strategies for Cross-Language Information Retrieval, Proceedings of the Third European Conference on Research and Advanced Technology for Digital Libraries, p.274-293, September 22-24, 1999
• 10: D. Hiemstra, F. de Jong, and W. Kraaij. A domain specific lexicon acquisition tool for cross-language information retrieval. In L. Devroye and C. Chrisment, editors, Proceedings of the Fifth RIAO International Conference, pages 255--270, Montréal, Canada, 1997. Centre de Hautes Études Internationales d'Informatique Documentaire (C.I.D).
• 11: John Lafferty , Chengxiang Zhai, Document language models, query models, and risk minimization for information retrieval, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.111-119, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383970]
• 12: Victor Lavrenko , W. Bruce Croft, Relevance based language models, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.120-127, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383972]
• 13: M. F. Porter. An algorithm for suffix stripping. Program, 14(3):130--137, July 1980.
• 14: S. E. Robertson. The probability ranking principle in IR. Journal of Documentation, 33:294--304, 1977. Reprinted in {16}.
• 15: Páraic Sheridan , Jean Paul Ballerini, Experiments in multilingual information retrieval using the SPIDER system, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.58-65, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243213]
• 16: Karen Sparck Jones , Peter Willett, Readings in information retrieval, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1997
• 17: E. M. Voorhees and D. K. Harman, editors. Proceedings of the Ninth Text REtrieval Conference (TREC-9), Gaithersburg, MD, November 2000. Department of Commerce, National Institute of Standards and Technology.
• 18: J. Xu and R. Weischedel. TREC-9 cross-lingual retrieval at BBN. In Voorhees and Harman {17}, pages 106--116.
• 19: Jinxi Xu , Ralph Weischedel , Chanh Nguyen, Evaluating a probabilistic model for cross-lingual information retrieval, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.105-110, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383968]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Experimentation

Keywords:
cross-language information retrieval, language models