Content-based video retrieval: does video's semantic visual feature matter?

ABSTRACT

A new shot level video browsing method based on semantic visual features (e.g., car, mountain, and fire) is proposed to facilitate content-based retrieval. The video's binary semantic feature vector is utilized to calculate the score of similarity between two shot keyframes. The score is then used to browse the "similar" keyframes in terms of semantic visual features. A pilot user study was conducted to better understand users' behaviors in video retrieval context. Three video retrieval and browsing systems are compared: temporal neighbor, semantic visual feature, and fused browsing system. The initial results indicated that the semantic visual feature browsing was effective and efficient for Visual Centric tasks, but not for Non-visual Centric tasks.

REFERENCES

• 1: Chang,S.-F., Hsu, W., Kennedy, L., Xie, L. , Yanagawa, A. , Zavesky, E., and Zhang, D.-Q., 2005, Columbia University TRECVID-2005 Video Search and High-Level Feature Extraction, in Proceedings of TRECVID2005
• 2: Heesch, D., Howarth, P., Magalhaes, J., May, A., Pickering, M., Yavlinsky, A., and ruger, S. (2004). Video retrieval using search and browsing. In proceedings of TRECVID2004.
• 3: Mezaris, Y., Doulaverakis, H., Herrmann, S., Lehane, B., O'Connor, N., Kompatsiaris, I., and Strintzis, G. M. (2004). Combining textual and visual information processing for interactive video retrieval: SCHEMA's participation to TRECVID2004. In proceedings of TRECVID2004 program.
• 4: Naphade, R. M., Kennedy, L., Kender, R. J. , chang, S., Smith, R. J. , Over, P., and Hauptmann, A. (2005). Alight scale concept ontology for multimedia understanding for TRECVID 2005.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Relevance feedback

General Terms:
Algorithms, Design, Experimentation

Keywords:
content-based, user interface, video browsing, video retrieval