Comparing cross-language query expansion techniques by degrading translation resources

ABSTRACT

The quality of translation resources is arguably the most important factor affecting the performance of a cross-language information retrieval system. While many investigations have explored the use of query expansion techniques to combat errors induced by translation, no study has yet examined the effectiveness of these techniques across resources of varying quality. This paper presents results using parallel corpora and bilingual wordlists that have been deliberately degraded prior to query translation. Across different languages, translingual resources, and degrees of resource degradation, pre-translation query expansion is tremendously effective. In several instances, pre-translation expansion results in better performance when no translations are available, than when an uncompromised resource is used without pre-translation expansion. We also demonstrate that post-translation expansion using relevance feedback can confer modest performance gains. Measuring the efficacy of these techniques with resources of different quality suggests an explanation for the conflicting reports that have appeared in the literature.

REFERENCES

• 1: Lisa Ballesteros , W. Bruce Croft, Phrasal translation and query expansion techniques for cross-language information retrieval, Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval, p.84-91, July 27-31, 1997, Philadelphia, Pennsylvania, United States
• 2: Lisa Ballesteros , W. Bruce Croft, Resolving ambiguity for cross-language retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.64-71, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290958]
• 3: Adam Berger , John Lafferty, Information retrieval as statistical translation, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.222-229, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312681]
• 4: C. Buckley, M. Mitra, J. Walz, and C. Cardie, 'Using Clustering and Super Concepts within SMART: TREC-6.' In E. Voorhees and D. Harman (eds.), Proceedings of the Sixth Text REtrieval Conference (TREC-6), NIST Special Publication 500--240, 1998.
• 5: C. Buckley, 'The TREC-9 Query Track.' In E. M. Voorhees and D. K. Harman, eds., Proceedings of the Ninth Text REtrieval Conference (TREC-9), pp. 81--85, 2001.
• 6: Kenneth Ward Church, Char_align: a program for aligning parallel texts at the character level, Proceedings of the 31st annual meeting on Association for Computational Linguistics, p.1-8, June 22-26, 1993, Columbus, Ohio [doi>10.3115/981574.981575]
• 7: A. Diekema, 'May the Best Team Win: Language Resources in CLIR.' Position paper at the CLEF-2000 workshop. Available online at: http://clef.iei.pi.cnr.it:2002/DELOS/CLEF/diekema.pdf
• 8: Martin Franz , J. Scott McCarley , Todd Ward , Wei-Jing Zhu, Quantifying the utility of parallel corpora, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.398-399, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.384037]
• 9: F. Gey and A. Chen, 'TREC-9 Cross-Language Information Retrieval (English - Chinese) Overview.' In E. M. Voorhees and D. K. Harman, eds., Proceedings of the Ninth Text REtrieval Conference (TREC-9), pp. 15--23, 2001.
• 10: Julio Gonzalo, Language Resources in Cross-Language Text Retrieval: A CLEF Perspective, Revised Papers from the Workshop of Cross-Language Evaluation Forum on Cross-Language Information Retrieval and Evaluation, p.36-47, September 21-22, 2000
• 11: Donna Harman, Relevance feedback revisited, Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, p.1-10, June 21-24, 1992, Copenhagen, Denmark [doi>10.1145/133160.133167]
• 12: D. Harman, 'Overview of the Fourth Text REtrieval Conference (TREC-4).' In D. K. Harman, ed., Proceedings of the Fourth Text REtrieval Conference (TREC-4), NIST Special Publication 500-236, pp. 1--24, 1995.
• 13: D. Hiemstra, 'Using Language Models for Information Retrieval.' Ph. D. Thesis, Center for Telematics and Information Technology, The Netherlands, 2000.
• 14: Wessel Kraaij, TNO at CLEF-2001: Comparing Translation Resources, Revised Papers from the Second Workshop of the Cross-Language Evaluation Forum on Evaluation of Cross-Language Information Retrieval Systems, p.78-93, September 03-04, 2001
• 15: K. L. Kwok , M. Chan, Improving two-stage ad-hoc retrieval for short queries, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.250-256, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291003]
• 16: T. K. Landauer and M. L. Littman, 'Fully automated cross-language document retrieval using latent semantic indexing.' In the Proceedings of the 6th Annual Conference of the UW Centre for the New Oxford English Dictionary and Text Research. 31--38, 1990.
• 17: James Mayfield , Paul McNamee, Converting on-line bilingual dictionaries from human-readable to machine-readable form, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564466]
• 18: Paul McNamee , James Mayfield, JHU/APL Experiments at CLEF: Translation Resources and Score Normalization, Revised Papers from the Second Workshop of the Cross-Language Evaluation Forum on Evaluation of Cross-Language Information Retrieval Systems, p.193-208, September 03-04, 2001
• 19: D. Oard and A. Diekema, 'Cross-Language Information Retrieval.' In M. Williams (ed.), Annual Review of Information Science, pp. 223--256, 1998.
• 20: C. Peters, 'Foreward to the Proceedings of the CLEF-2001 Workshop', to appear in 2002.
• 21: Ari Pirkola , Turid Hedlund , Heikki Keskustalo , Kalervo Järvelin, Dictionary-Based Cross-Language Information Retrieval: Problems, Methods, and Research Findings, Information Retrieval, v.4 n.3-4, p.209-230, September-December 2001 [doi>10.1023/A:1011994105352]
• 22: Jay M. Ponte , W. Bruce Croft, A language modeling approach to information retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.275-281, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291008]
• 23: Yonggang Qiu , Hans-Peter Frei, Concept based query expansion, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.160-169, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States [doi>10.1145/160688.160713]
• 24: Philip Resnik, Mining the Web for bilingual text, Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics, p.527-534, June 20-26, 1999, College Park, Maryland [doi>10.3115/1034678.1034757]
• 25: G. Salton and C. Buckley, 'Improving Retrieval Performance by Relevance Feedback.' In the Journal of the American Society for Information Science, 41(4), pp. 288--297, 1990.
• 26: Jinxi Xu , W. Bruce Croft, Query expansion using local and global document analysis, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.4-11, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243202]
• 27: Jinxi Xu , Ralph Weischedel, Cross-lingual information retrieval using hidden Markov models, Proceedings of the 2000 Joint SIGDAT conference on Empirical methods in natural language processing and very large corpora: held in conjunction with the 38th Annual Meeting of the Association for Computational Linguistics, p.95-103, October 07-08, 2000, Hong Kong [doi>10.3115/1117794.1117806]
• 28: J. Xu, A. Fraser, and R. Weischedel, 'TREC 2001 Cross-lingual Retrieval at BBN.' In TREC-2001 Notebook Papers, pp. 122--131, 2001.
• 29: Cross-Language Evaluation Forum, http://www.clef-campaign.org/
• 30: NTCIR Project, http://research.nii.ac.jp/ntcir/
• 31: Text REtrieval Conference, http://trec.nist.gov/
• 32: http://dictionaries.travlang.com/
• 33: http://europa.eu.int/
• 34: http://www.june29.com/IDP/

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Query formulation

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Relevance feedback

General Terms:
Experimentation

Keywords:
cross-language information retrieval, query expansion, query translation, translation resources