Merging storyboard strategies and automatic retrieval for improving interactive video search

ABSTRACT

The Carnegie Mellon University Informedia group has enjoyed consistent success with TRECVID interactive search using traditional storyboard interfaces for shot-based retrieval. For TRECVID 2006 the output of automatic search was included for the first time with storyboards, both as an option for an interactive user and in a different run as the sole means of access. The automatic search makes use of relevance-based probabilistic retrieval models to determine weights for combining retrieval sources when addressing a given topic. Storyboard-based access using automatic search output outperformed extreme video retrieval interfaces of manual browsing with resizable pages and rapid serial visualization of keyframes that used the same output. Further, the full Informedia interface with automatic search results as an option along with other query mechanisms scored significantly better than all other TRECVID 2006 interactive search systems. Attributes of the automatic search and interactive search systems are discussed to further optimize shot-based retrieval from news corpora.

REFERENCES

• 1: Campbell, M., Ebadollahi, S., Naphade, M., Natsev, A., Smith, J. R., Tesic, J., Xie, L., Haubold, A. IBM Research TRECVID-2006 Video Retrieval System. In NIST TREC Video Retrieval Online Proceedings. NIST, 2006, //www-nlpir.nist.gov/projects/tvpubs/tv6.papers/ibm.pdf.
• 2: Michael Christel , Neema Moraveji, Finding the right shots: assessing usability and performance of a digital video library interface, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027691]
• 3: Christel, M., and Conescu, R. Mining Novice User Activity with TRECVID Interactive Retrieval Tasks. In LNCS 4071: Proc. Image and Video Retrieval (CIVR) (Tempe, AZ, July 2006), Springer, Berlin, 2006, 21--30.
• 4: Chua, T.-S., Neo, S.-Y., Li, K.-Y., Wang, G., Shi, R., Zhao, M., Xu, H. TRECVID 2004 Search and Feature Extraction Task by NUS PRIS. In NIST TREC Video Retrieval Online Proceedings. NIST, Gaithersburg, MD, 2004, http://www-nlpir.nist.gov/projects/tvpubs/tvpapers04/nus.pdf.
• 5: Alexander G. Hauptmann , Wei-Hao Lin , Rong Yan , Jun Yang , Ming-Yu Chen, Extreme video retrieval: joint maximization of human and computer performance, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180721]
• 6: Kraaij, W., Over, P., Ianeva, T., and Smeaton, A. TRECVID 2006 - An Introduction. In TRECVID Online Proceedings, //www-nlpir.nist.gov/projects/tvpubs/tv6.papers/tv6intro.pdf.
• 7: Milind Naphade , John R. Smith , Jelena Tesic , Shih-Fu Chang , Winston Hsu , Lyndon Kennedy , Alexander Hauptmann , Jon Curtis, Large-Scale Concept Ontology for Multimedia, IEEE MultiMedia, v.13 n.3, p.86-91, July 2006 [doi>10.1109/MMUL.2006.63]
• 8: Snoek, C., Worring, M., Koelma, D., and Smeulders, A. Learned Lexicon-Driven Interactive Video Retrieval. In LNCS 4071: Proc. Image and Video Retrieval (CIVR) (Tempe, AZ, July 2006), Springer, Berlin, 2006, 11--20.
• 9: Snoek, C., Worring, M., Koelma, D., and Smeulders, A. A Learned Lexicon-Driven Paradigm for Interactive Video Retrieval. IEEE Trans. Multimedia 9(2), Feb. 2007, 280--292.
• 10: Rong Yan , Jun Yang , Alexander G. Hauptmann, Learning query-class dependent weights in automatic video retrieval, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027661]
• 11: Yan, R., and Hauptmann, A. G. Efficient margin-based rank learning algorithms for information retrieval. In LNCS 4071: Proc. Image and Video Retrieval (CIVR) (Tempe, AZ, July 2006), Springer, Berlin, 2006, 113--122.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Experimentation, Human Factors

Keywords:
TRECVID results analysis, video retrieval