A color fingerprint of video shot for content identification

ABSTRACT

In this paper we propose a novel space-time color feature representation for video shot and apply it to content identification. In this representation the shot is cut into <i>k</i> equal size segments, and each segment is represented by a blending image formed through averaging the pixels' values of each frame in this segment along time direction. Each blending image is then divided into equal size blocks, and two color patterns named major and minor colors among mean R,G,B are extracted for each block. Hence each shot can be represented by a fixed-length string. The experiment shows this representation is not only robust to image quality reduction, frame size and frame rate change, but also to color distortion such as brightness/contrast adjustment. We also give a video similarity measure based on this color feature to identify shot chunks. We conducted experiment on 100 video clips, and quite low error rates can be achieved when identifying small size shot chunks with significant color distortion. From the experiment we believe that this color feature is a compact and robust representation for video content, and effective for content identification.

REFERENCES

• 1: Donald A. Adjeroh , M. C. Lee , Irwin King, A distance measure for video sequences, Computer Vision and Image Understanding, v.75 n.1-2, p.25-45, July/Aug. 1999 [doi>10.1006/cviu.1999.0764]
• 2: Liping Chen and Tat-Seng Chua, A match and tiling approach to content-based image retrieval, IEEE Int'l Conference on Multimedia and Expo (Tokyo, Japan), August 2001, pp. 417--420.
• 3: A. Mofit Ferman, A. MuratTekalp, and Rajiv Mehrotra, Robust color histogram descriptors for video segment retrieval and identification, IEEE Trans. on Image Processing 11 (2002), no. 5, 497--508.
• 4: Arun Hampapur and Ruud Bolle, Comparison of distance measures for video copy detection, International Conference on Multimedia and Expo, 2001.
• 5: Yong Rui , Thomas S. Huang , Sharad Mehrotra, Exploring Video Structure Beyond The Shots, Proceedings of the IEEE International Conference on Multimedia Computing and Systems, p.237, June 28-July 01, 1998 [doi>10.1109/MMCS.1998.693648]
• 6: Juan M. Sánchez , Xavier Binefa , Jordi Vitrià, Shot Partitioning Based Recognition of TV Commercials, Multimedia Tools and Applications, v.18 n.3, p.233-247, December 2002 [doi>10.1023/A:1019996817159]
• 7: W. Wolf, Key frame selection by motion analysis, Proc. ICASSP, vol. II, 1996, pp. 1228--1231.
• 8: Xianfeng Yang, Qi Tian, and Sheng Gao, Video clip representation and recognition using composite shot models, ICICS-PCM, 2003.
• 9: H. J. Zhang, J.Wu, D. Zhong, and S.W. Somaliar, An integrated system for content-based video retrieval and browsing, Pattern Recognition 30 (1997), no. 4, 643--658.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Abstracting methods

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.7 Feature Measurement
          Subjects: Feature representation

General Terms:
Algorithms, Design, Experimentation

Keywords:
color fingerprint, shot representation, video content identification