Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

In this paper we present the results of our work on the analysis of multi-modal data for video Information Retrieval, where we exploit the properties of this data for query-time, automatic generation of weights for multi-modal data fusion. Through empirical testing we have observed that for a given topic, a high performing feature,that is one which achieves high relevance, will have a different distribution of document scores when compared against those that do not perform as well. These observations form the basis for our initial fusion model, which generates weights based on these properties, without the need for prior training.Our model can be used to not only combine feature data,but to also combine the results of multiple example query images and apply weights to these.Our analysis and experiments were conducted on the TRECVid 2004 and 2005 collections,making use of multiple MPEG-7 low-level features and automatic speech recognition (ASR)transcripts.Results achieved from our model achieve performance on a par with that of 'oracle' determined weights,and demonstrate the applicability of our model whilst advancing the case for further investigation of score distributions.

REFERENCES

• 1: The AceMedia Project,available at http://www.acemedia.org.
• 2: Learning of personal visual impressions for image database systems.In In Proceedings of the Second International Conference on Document Analysis and Recognition pages 547--552, 1993.
• 3: Brian T. Bartell , Garrison W. Cottrell , Richard K. Belew, Automatic combination of multiple ranked retrieval systems, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.173-181, July 03-06, 1994, Dublin, Ireland
• 4: S.-F.Chang, T. Sikora, and A. Puri. Overview of the MPEG-7 standard.IEEE Transactions on Circuits and Systems for Video Technology 11(6): 688--695, June 2001.
• 5: E. Cooke, P. Ferguson, G. Gaughan, C. Gurrin, G. J. F. Jones, H. L. Borgne, H. Lee, S. Marlow, K. McDonald, M. McHugh, N. Murphy, N. E. O 'Connor, N.O 'Hare, S. Rothwell, A. F. Smeaton, and P. Wilkins. TRECVID 2004 experiments in Dublin City University. In Proceedings of TRECVID 2004 November 2004.
• 6: A. P. Dempster. A generalization of the Bayesian inference. Journal of Royal Statistical Society 30:205--447, 1968.
• 7: P. Ferguson, C. Gurrin, P. Wilkins, and A. F. Smeaton.Físréál: A Low Cost Terabyte Search Engine. In Proceedings of European Conference in IR March 2005.
• 8: E. A. Fox and J. A. Shaw. Combination of multiple searches. In Proceedings of the 2nd Text REtrieval Conference 1994.
• 9: Andrew Graves , Mounia Lalmas, Video retrieval using an MPEG-7 based inference network, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564436]
• 10: Joemon M. Jose , Jonathan Furner , David J. Harper, Spatial querying for image retrieval: a user-oriented evaluation, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.232-240, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291000]
• 11: B. S. Manjunath, J. R. Ohm, V. Vasudevan, and A. Yamada. Color and texture description. In IEEE Trans. On Circuits and Systems for Video Technology June 2001.
• 12: R. Manmatha , T. Rath , F. Feng, Modeling score distributions for combining the outputs of search engines, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.267-275, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.384005]
• 13: R. W. Picard, T. P. Minka, and M. Szummer. Modeling user subjectivity in image libraries. In IEEE International Conference On Image Processing 1996.
• 14: Vassilis Plachouras , Iadh Ounis, Dempster-Shafer Theory for a Query-Biased Combination of Evidence on the Web, Information Retrieval, v.8 n.2, p.197-218, April 2005 [doi>10.1007/s10791-005-5659-7]
• 15: S. E. Robertson , S. Walker, Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.232-241, July 03-06, 1994, Dublin, Ireland
• 16: Y. Rui and T. Huang. Optimizing learning in image retrieval.In Proceeding of IEEE int. Conf. On Computer Vision and Pattern Recognition pages 236--245, 2000.
• 17: Y. Rui, T. S. Huang, M. Ortega, and S. Mehrotra. Relevance feedback:A power tool for interactive content-based image retrieval relevance feedback: A power tool for interactive content-based image retrieval. IEEE Transactions on Circuits and Systems for Video Technology 8(5): 644--655 1998.
• 18: Gerald Salton, Automatic text processing, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1988
• 19: G. Shafer. A Mathematical Theory of Evidence. Princeton University Press 1976.
• 20: A. F. Smeaton. Large Scale Evaluations of Multimedia Information Retrieval: The TRECVid Experience volume 3568 /2006,pages 11--17.Springer, 2005.
• 21: Howard Turtle , W. Bruce Croft, Evaluation of an inference network-based retrieval model, ACM Transactions on Information Systems (TOIS), v.9 n.3, p.187-222, July 1991 [doi>10.1145/125187.125188]
• 22: P. Wilkins, P. Ferguson, C. Gurrin, and A. F. Smeaton. Automatic determination of feature weights for mult-feature CBIR. In ECIR 2006 - European Conference on Information Retrieval. Lalmas M et al. (Eds.): (LNCS Series 3936) pages 527--530. Springer,

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Experimentation, Measurement

Keywords:
automatic weight determination, multimedia fusion