Distribution-based concept selection for concept-based video retrieval

ABSTRACT

Query-to-concept mapping plays one of the keys to concept-based video retrieval. Conventional approaches try to find concepts that are likely to co-occur in the relevant shots from the lexical or statistical aspects. However, the high probability of co-occurrence alone cannot ensure its effectiveness to distinguish the relevant shots from the irrelevant ones. In this paper, we propose distribution based concept selection (DBCS) for query-to-concept mapping by analyzing concept score distributions of within and between relevant and irrelevant sets. In view of the imbalance between relevant and irrelevant examples, two variants of DBCS are proposed respectively by considering the two-sided and onesided metrics of concept distributions. Specifically, the impact of positive and negative concepts toward search is explicitly considered. DBCS is found to be appropriate for both automatic and interactive video search. Using TRECVID 2008 video dataset for experiments, improvements of 50% and 34% are reported when compared to text-based and visual-based query-to concept mapping respectively in automatic search. Meanwhile, DBCS shows continuous improvement for all rounds of user feedbacks in interactive search.

REFERENCES

• 1: A. Hauptmann, Y. Rong, W.H. Lin, M. Christel, H.Wactlar, "Can High-Level Concepts Fill the Semantic Gap in Video Retrieval? A Case Study With Broadcast News", IEEE Transactions on Multimedia, Vol. 9(5), pp: 958--966, 2007
• 2: P. Over, G. Awad, T. Rose, J. Fiscus, W. Kraaij, A.F. Smeaton, TRECVID 2008-Goals, Tasks, Data, Evaluation Mechanisms and Metrics, in Proceedings of TRECVID Workshop, USA, 2008
• 3: A. Natsev, A. Haubold, J. Tesic, L. Xie, and R. Yan. Semantic concept-based query expansion and re-ranking for multimedia retrieval: A comparative review and new approaches. In ACM Multimedia (ACM MM), Sep. 2007.
• 4: Y. Lu, L. Zhang, Q. Tian, W.Y. Ma. What Are the High level Concepts with Small Semantic Gaps. IEEE Conference on CVPR, pp.1--8, 2008
• 5: B. Huurnink, K. Hofmann, Maarten de Rijke Assessing Concept Selection for Video Retrieval, MIR'08, pp. 459--466
• 6: Y.M. Yang and O. Pedersen. A Comparative Study on Feature Selection in Text Categorization. Proceedings of ICML-97, pp. 412--420, 1997
• 7: Y.G. Jiang, A. Yanagawa, S.F. Chang, and C.W.Ngo, "CUVIREO374: Fusing Columbia374 and VIREO374 for Large Scale Semantic Concept Detection", Columbia University ADVENT Technical Report #223-2008-1, 2008.
• 8: A. Natsev, M. R. Naphade, and J. Tesic. Learning the semantics of multimedia queries and concepts from a small number of examples. In ACM Multimedia , Nov. 6--11 2005.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Experimentation, Performance

Keywords:
concept-based video retrieval, distribution, query-to-concept mapping