Learning query-class dependent weights in automatic video retrieval

ABSTRACT

Combining retrieval results from multiple modalities plays a crucial role for video retrieval systems, especially for automatic video retrieval systems without any user feedback and query expansion. However, most of current systems only utilize query independent combination or rely on explicit user weighting. In this work, we propose using query-class dependent weights within a hierarchial mixture-of-expert framework to combine multiple retrieval results. We first classify each user query into one of the four predefined categories and then aggregate the retrieval results with query-class associated weights, which can be learned from the development data efficiently and generalized to the unseen queries easily. Our experimental results demonstrate that the performance with query-class dependent weights can considerably surpass that with the query independent weights.

REFERENCES

• 1: A. Amir, M. Berg, S.-F. Chang, and et al. IBM research TRECVID-2003 video retrieval system. In NIST TRECVID-2003, Nov 2003.
• 2: M. Beigi, A. B. Benitez, and S.-F. Chang. Metaseek: A content-based meta-search engine for images. In Proc. of SPIE, 1997.
• 3: Daniel M. Bikel , Scott Miller , Richard Schwartz , Ralph Weischedel, Nymble: a high-performance learning name-finder, Proceedings of the fifth conference on Applied natural language processing, p.194-201, March 31-April 03, 1997, Washington, DC [doi>10.3115/974557.974586]
• 4: Eric Brill, Some advances in transformation-based part of speech tagging, Proceedings of the twelfth national conference on Artificial intelligence (vol. 1), p.722-727, October 1994, Seattle, Washington, United States
• 5: P. Clarkson and R. Rosenfeld. Statistical language modeling using the CMU-Cambridge toolkit. In Proc. Eurospeech'97, 1997.
• 6: Michael Collins , Robert E. Schapire , Yoram Singer, Logistic Regression, AdaBoost and Bregman Distances, Proceedings of the Thirteenth Annual Conference on Computational Learning Theory, p.158-169, June 28-July 01, 2000
• 7: A. Dempster, N. Laird, and D. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39(1):1--38, 1977.
• 8: Georgina Gaughan , Alan F. Smeaton , Cathal Gurrin , Hyowon Lee , Kieran McDonald, Design, implementation and testing of an interactive video retrieval system, Proceedings of the 5th ACM SIGMM international workshop on Multimedia information retrieval, November 07-07, 2003, Berkeley, California [doi>10.1145/973264.973269]
• 9: D. Grinberg, J. Lafferty, and D. Sleator. A robust parsing algorithm for link grammars. In Proc. of the 4th Int'l Workshop on Parsing Technologies, 1995.
• 10: A. G. Hauptmann and et al. Informedia at TRECVID 2003: Analyzing and searching broadcast news video. In Proc. of TRECVID 2003, Gaithersburg, MD, 2003.
• 11: Michael I. Jordan , Robert A. Jacobs, Hierarchical mixtures of experts and the EM algorithm, Neural Computation, v.6 n.2, p.181-214, March 1994 [doi>10.1162/neco.1994.6.2.181]
• 12: In-Ho Kang , GilChang Kim, Query type classification for web document retrieval, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860449]
• 13: Xin Li , Dan Roth, Learning question classifiers, Proceedings of the 19th international conference on Computational linguistics, p.1-7, August 24-September 01, 2002, Taipei, Taiwan [doi>10.3115/1072228.1072378]
• 14: Andrew Merlino , Daryl Morey , Mark Maybury, Broadcast news navigation using story segmentation, Proceedings of the fifth ACM international conference on Multimedia, p.381-391, November 09-13, 1997, Seattle, Washington, United States [doi>10.1145/266180.266390]
• 15: L. Ramshaw and M. Marcus. Text chunking using transformation-based learning. In Proc. of the ACL Third Workshop on Very Large Corpora, 1995.
• 16: S. E. Robertson, S. Walker, M. Hancock-Beaulieu, A. Gull, and M. Lau. Okapi at TREC4. In Text REtrieval Conference, pages 21--30, 1992.
• 17: Gerard Salton , Michael J. McGill, Introduction to Modern Information Retrieval, McGraw-Hill, Inc., New York, NY, 1986
• 18: TRECVID: TREC Video Retrieval Evaluation. http://www-nlpir.nist.gov/projects/trecvid.
• 19: T. Westerveld, T. Ianeva, L. Boldareva, A. P. de Vries, and D. Hiemstra. Combining infomation sources for video retrieval: The lowlands team at TRECVID 2003. In NIST TRECVID-2003, Nov 2003.
• 20: R. Yan and A. Hauptmann. Co-retrieval: A boosted reranking approach for multimedia retrieval. In Proc. of Intl. Conf. on Image and Video Retrieval, 2004.
• 21: Hui Yang , Lekha Chaisorn , Yunlong Zhao , Shi-Yong Neo , Tat-Seng Chua, VideoQA: question answering on news video, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957146]
• 22: Yiming Yang , Jan O. Pedersen, A Comparative Study on Feature Selection in Text Categorization, Proceedings of the Fourteenth International Conference on Machine Learning, p.412-420, July 08-12, 1997
• 23: Dell Zhang , Wee Sun Lee, Question classification using support vector machines, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860443]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

General Terms:
Algorithms, Performance, Theory

Keywords:
learning, modality fusion, query class, video retrieval