Adaptive multiple feedback strategies for interactive video search

ABSTRACT

In this paper, we propose adaptive multiple feedback strategies for interactive video retrieval. We first segregate interactive feedback into 3 distinct types (recall-driven relevance feedback, precision-driven active learning and locality-driven relevance feedback) so that a generic interaction mechanism with more flexibility can be performed to cover different search queries and different video corpuses. Our system facilitates expert searchers to flexibly decide on the types of feedback they want to employ under different situations. To cater to the large number of novice users (non-expert users), an adaptive option is built-in to learn the expert user behavior so as to provide recommendations on the next feedback strategy, leading to a more precise and personalized search for the novice users. Experimental results on TRECVID news video corpus demonstrate that our proposed adaptive multiple feedback strategies are effective.

REFERENCES

• 1: Christel M., Huang C., Moraveji N., Papernick N.: Exploiting multiple modalities for interactive video retrieval. In: Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, pp. 1032--1035, Montreal, Canada, 2004.
• 2: Snoek C. G. M., Worring M., Koelma D. C., Smeulders A. W. M.: A learned lexicon-driven paradigm for interactive video retrieval. IEEE Trans. on Multimedia, 9(2):280--292, 2007.
• 3: Yong Rui , Thomas S. Huang , Sharad Mehrotra , Michael Ortega, A Relevance Feedback Architecture for Content-based Multimedia Information Retrieval Systems, Proceedings of the 1997 Workshop on Content-Based Access of Image and Video Libraries (CBAIVL '97), p.82, June 20-20, 1997
• 4: Rui Y., Huang T. S., Mehrotra S.: Content-based image retrieval with relevance feedback in MARS. In: Proc. IEEE Int. Conf. on Image Processing, pp.815--818, USA, 1997.
• 5: Wei C. H., Li C. T.: Content-based multimedia retrieval. In: Proc. Encyclopedia of Multimedia Technology and Networking, pp.116--122, Hershey, PA, USA, 2005.
• 6: Crucianu M., Ferecatu M., Boujemaa N.: Relevance feedback for image retrieval: A short survey. In: State of the Art in Audiovisual Content-Based Retrieval, Information Universal Access and Interaction, 2004.
• 7: Guo G., Jain A.K., Ma W., Zhang H.: Learning similarity measure for natural image retrieval with relevance feedback. IEEE Trans. Neural Networks, 12(4):811--820, 2002.
• 8: Hong P., Tian Q., Huang T. S..: Incorporate support vector machines to content-based image retrieval with relevant feedback. In: Proc. IEEE Int. Conf. Image Processing, pp.750--753, Vancouver, Canada, 2000.
• 9: Tao D., Tang X.: Random sampling based SVM for relevance feedback image retrieval. In: Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition, pp.647--652, Washington, DC, USA, 2004.
• 10: Zhang L., Lin F., Zhang B.: Support vector machine learning for image retrieval. In: Proc. IEEE Int. Conf. Image Processing, pp.721--724, Thessaloniki, Greece, 2001.
• 11: Dacheng Tao , Xiaoou Tang , Xuelong Li , Xindong Wu, Asymmetric Bagging and Random Subspace for Support Vector Machines-Based Relevance Feedback in Image Retrieval, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.28 n.7, p.1088-1099, July 2006 [doi>10.1109/TPAMI.2006.134]
• 12: Steven C. H. Hoi , Michael R. Lyu, A Semi-Supervised Active Learning Framework for Image Retrieval, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2, p.302-309, June 20-26, 2005 [doi>10.1109/CVPR.2005.44]
• 13: Szummer M., Jaakkola T.: Partially labeled classification with markov random walks. In: Proc. Advances in Neural Information Processing Systems, pp.945--952, Vancouver, Canada, 2001.
• 14: Steven C. H. Hoi , Rong Jin , Michael R. Lyu, Large-scale text categorization by batch mode active learning, Proceedings of the 15th international conference on World Wide Web, May 23-26, 2006, Edinburgh, Scotland [doi>10.1145/1135777.1135870]
• 15: Simon Tong , Edward Chang, Support vector machine active learning for image retrieval, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500159]
• 16: Alexander G. Hauptmann , Wei-Hao Lin , Rong Yan , Jun Yang , Ming-Yu Chen, Extreme video retrieval: joint maximization of human and computer performance, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180721]
• 17: Ork de Rooij , Cees G. M. Snoek , Marcel Worring, Query on demand video browsing, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291417]
• 18: Huan-Bo Luan , Shi-Yong Neo , Hai-Kiat Goh , Yong-Dong Zhang , Shou-Xun Lin , Tat-Seng Chua, Segregated feedback with performance-based adaptive sampling for interactive news video retrieval, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291295]
• 19: Chua T. S., Neo S. Y., Zheng Y. T., Goh H. K., Xiao Y., Zhao M.: TRECVID 2006 by NUS-I2R. In: Proc. TRECVID, Gaithersburg, USA, 2006.
• 20: Chua T. S., Neo S. Y., Goh H. K., Zhao M., Xiao Y., Wang G.: TRECVID 2005 by NUS PRIS. In: Proc. TRECVID, Gaithersburg, USA, 2005.
• 21: Robertson S.E. and Sparck Jones K, "Relevance Weighting of Search Terms," Journal of the American Society for Information Science, 27(3), 129--146, 1976.
• 22: Michael G. Christel , Rong Yan, Merging storyboard strategies and automatic retrieval for improving interactive video search, Proceedings of the 6th ACM international conference on Image and video retrieval, p.486-493, July 09-11, 2007, Amsterdam, The Netherlands [doi>10.1145/1282280.1282351]
• 23: Bernhard Schölkopf and Alex Smola: Learning with Kernels. MIT Press, Cambridge, MA, USA, 2002.
• 24: http://www-nlpir.nist.gov/projects/t01v/

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Relevance feedback

General Terms:
Algorithms, Design, Experimentation

Keywords:
adaptive model, interactive video search, relevance feedback