Evaluating a probabilistic model for cross-lingual information retrieval

ABSTRACT

This work proposes and evaluates a probabilistic cross-lingual retrieval system. The system uses a generative model to estimate the probability that a document in one language is relevant, given a query in another language. An important component of the model is translation probabilities from terms in documents to terms in a query. Our approach is evaluated when 1) the only resource is a manually generated bilingual word list, 2) the only resource is a parallel corpus, and 3) both resources are combined in a mixture model. The combined resources produce about 90% of monolingual performance in retrieving Chinese documents. For Spanish the system achieves 85% of monolingual performance using only a pseudo-parallel Spanish-English corpus. Retrieval results are comparable with those of the structural query translation technique (Pirkola, 1998) when bilingual lexicons are used for query translation. When parallel texts in addition to conventional lexicons are used, it achieves better retrieval results but requires more computation than the structural query translation technique. It also produces slightly better results than using a machine translation system for CLIR, but the improvement over the MT system is not significant.

REFERENCES

• 1: Allan, J., Callan, J., Feng, F-F, and Malin, D. 2000. "INQUERY at TREC8." In TREC8 Proceedings, Special publication by NIST, 2000.
• 2: Lisa Ballesteros , W. Bruce Croft, Resolving ambiguity for cross-language retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.64-71, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290958]
• 3: Adam Berger , John Lafferty, Information retrieval as statistical translation, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.222-229, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312681]
• 4: Peter F. Brown , Vincent J. Della Pietra , Stephen A. Della Pietra , Robert L. Mercer, The mathematics of statistical machine translation: parameter estimation, Computational Linguistics, v.19 n.2, June 1993
• 5: Djoerd Hiemstra , Franciska de Jong, Disambiguation Strategies for Cross-Language Information Retrieval, Proceedings of the Third European Conference on Research and Advanced Technology for Digital Libraries, p.274-293, September 22-24, 1999
• 6: David Hull, Using statistical testing in the evaluation of retrieval experiments, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.329-338, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States [doi>10.1145/160688.160758]
• 7: Hull, D. 1997. "Using structured queries for disambiguation in cross-language information retrieval." In AAAI Symposium on Cross-Language Text and Speech Retrieval, 1997.
• 8: Klavans, J. and Hovy, E. 1999. "Multilingual (or Crosslingual) Information Retrieval". Chapter 2, Multilingual Information Management, current levels and future abilities. Editors, E. Hovy, N. Ide, R. Frederking, J. Mariani and A. Zampolli, Arpil, 1999.
• 9: K. L. Kwok, Comparing representations in Chinese information retrieval, Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval, p.34-41, July 27-31, 1997, Philadelphia, Pennsylvania, United States
• 10: Kwok, K.L. 2000. "TREC9 Cross-language, questionanswering track experiments using PIRCS." TREC9 Proceedings published by NIST, 2000.
• 11: Lafferty, J. 1999. Personal communications.
• 12: Levow, G.A. and Oard, D. 1999. "Evaluating lexical coverage for cross-language information retrieval." In Workshop on Multilingual Information Processing and Asian Language Processing, Beijing, 1999.
• 13: J. Scott McCarley, Should we translate the documents or the queries in cross-language information retrieval?, Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, p.208-214, June 20-26, 1999, College Park, Maryland [doi>10.3115/1034678.1034716]
• 14: David R. H. Miller , Tim Leek , Richard M. Schwartz, A hidden Markov model information retrieval system, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.214-221, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312680]
• 15: Douglas W. Oard, A Comparative Study of Query and Document Translation for Cross-Language Information Retrieval, Proceedings of the Third Conference of the Association for Machine Translation in the Americas on Machine Translation and the Information Soup, p.472-483, October 28-31, 1998
• 16: Ari Pirkola, The effects of query structure and dictionary setups in dictionary-based cross-language information retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.55-63, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290957]
• 17: Jay M. Ponte , W. Bruce Croft, A language modeling approach to information retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.275-281, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291008]
• 18: Porter, M. 1980. "An algorithm for suffix stripping." Program 14, 3(1980), pages 130-137.
• 19: Rabiner, L. 1989. "A tutorial on Hidden Markov models and selected applications in speech recognition", In Proceedings of IEEE 77, pages 257-286, 1989.
• 20: Amit Singhal , Chris Buckley , Mandar Mitra, Pivoted document length normalization, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.21-29, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243206]
• 21: Ruth Sperer , Douglas W. Oard, Structured translation for cross-language information retrieval, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.120-127, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345562]
• 22: Voorhees, E. and Harman, D. 1997. TREC-5 Proceedings. NIST special publication, 1997.
• 23: Voorhees, E. and Harman, D. 2000. TREC-9 Proceedings. To be published by NIST.
• 24: Jinxi Xu , W. Bruce Croft, Improving the effectiveness of information retrieval with local context analysis, ACM Transactions on Information Systems (TOIS), v.18 n.1, p.79-112, Jan. 2000 [doi>10.1145/333135.333138]

INDEX TERMS

Primary Classification:
  F. Theory of Computation
  F.1 COMPUTATION BY ABSTRACT DEVICES
      F.1.2 Modes of Computation
          Subjects: Probabilistic computation

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Linguistic processing
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Design, Experimentation, Performance