Experimental result analysis for a generative probabilistic image retrieval model

ABSTRACT

The main conclusion from the metrics-based evaluation of video retrieval systems at TREC's video track is that non-interactive image retrieval from general collections using visual information only is not yet feasible. We show how a detailed analysis of retrieval results -- looking beyond mean average precision (MAP) scores on topical relevance -- gives significant insight in the main problems with the visual part of the retrieval model under study. Such an analytical approach proves an important addition to standard evaluation measures.

REFERENCES

• 1: D. Hiemstra. Using language models for information retrieval. PhD thesis, Centre for Telematics and Information Technology, University of Twente, 2001.
• 2: Jay M. Ponte , W. Bruce Croft, A language modeling approach to information retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.275-281, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291008]
• 3: A. F. Smeaton and P. Over. The trec-2002 video track report. In E. M. Voorhees and D. K. Harman, editors, The Eleventh Text REtrieval Conference (TREC-2002), 2002.
• 4: Alan F. Smeaton , Paul Over , Cash Costello , Arjen P. de Vries , David S. Doermann , Alexander G. Hauptmann , Mark E. Rorvig , John R. Smith , Lide Wu, The TREC2001 Video Track: Information Retrieval on Digital Video Information, Proceedings of the 6th European Conference on Research and Advanced Technology for Digital Libraries, p.266-275, September 16-18, 2002
• 5: Nuno Miguel Borges De Pinho Cruz De Vasconcelos , Andrew B. Lippman, Bayesian models for visual information retrieval, 2000
• 6: Thijs Westerveld, Probabilistic multimedia retrieval, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564482]
• 7: T. Westerveld, A. P. de~Vries, and A. van Ballegooij. CWI at the TREC-2002 Video Track. In E. M. Voorhees and D. K. Harman, editors, The Eleventh Text REtrieval Conference (TREC-2002), 2002.
• 8: T. Westerveld, A. P. de~Vries, A. van Ballegooij, F. M. G. de~Jong, and D. Hiemstra. A probabilistic multimedia retrieval model and its evaluation. EURASIP Journal on Applied Signal Processing, 2003(2):186--198, 2003. special issue on Unstructured Information Management from Multimedia Data Sources.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Image databases

General Terms:
Experimentation

Keywords:
gaussian mixture models, multimedia retrieval, results analysis