The impact of evaluation on multilingual text retrieval

ABSTRACT

We summarize the impact of the first five years of activity of the Cross-Language Evaluation Forum (CLEF) on multilingual text retrieval system performance and show how the CLEF evaluation campaigns have contributed to advances in the state-of-the-art.

REFERENCES

• 1: Peters, C., Braschler, M., Di Nunzio, G., Ferro, N. CLEF 2004: Ad Hoc Track Overview and Results Analysis. LNCS 3491, (2005), forthcoming.
• 2: Braschler, M. CLEF 2003 Overview of Results, LNCS 3237, (2004), 44--63.
• 3: Martínez, F., Ureña, L. and Martín, T. Experiments with merging strategies. LNCS 2785 (2003), 187--196.
• 4: See CLEF website: http://clef.isti.cnr.it/publications.html
• 5: Martin Braschler , Carol Peters, Cross-Language Evaluation Forum: Objectives, Results, Achievements, Information Retrieval, v.7 n.1-2, p.7-31, January-April 2004 [doi>10.1023/B:INRT.0000009438.69013.fa]
• 6: McNamee, P. and Mayfield, J. Scalable Multilingual Information Access. LNCS 2785 (2003), 207--218.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software
          Subjects: Performance evaluation (efficiency and effectiveness)

General Terms:
Experimentation, Languages, Measurement, Performance

Keywords:
multilingual information access, system evaluation