Video keyframe production by efficient clustering of compressed chromaticity signatures (poster session)

ABSTRACT

We develop a new low-dimensional video frame feature that is more insensitive to lighting change, motivated by color constancy work in physics-based vision, and apply the feature to keyframe production using hierarchical clustering. The new feature has the further advantage of more expressively capturing image information and as a result produces a very succinct set of keyframes for any video. Because we effectively reduce any video to the same lighting conditions, we can produce a universal basis on which to project video frame features. We carry out clustering efficiently by adapting a hierarchical clustering data structure to temporally-ordered clusters. Using a new multi-stage hierarchical clustering method, we merge clusters based on the ratio of cluster variance to variance of the parent node, merging only adjacent clusters, and then follow with a second round of clustering. The second stage merges clusters incorrectly split in the first round by the greedy hierarchical algorithm, and as well merges non-adjacent clusters to fuse near-repeat shots. The new summarization method produces a very succinct set of keyframes for videos, and results are excellent.

REFERENCES

• 1: B.-L. Yeo and M.M. Yeung. Classification, simplification and dynamic visualization of scene transition graphs for video browsing. In SPIE Storage and Retrieval for Image and Video Databases VI, 1998.
• 2: M. M. Yeung , Bede Liu, Efficient matching and clustering of video shots, Proceedings of the 1995 International Conference on Image Processing (Vol. 1)-Volume 1, p.338, October 23-26, 1995
• 3: D. Zhong, H. Zhang, and S.-F. Chang. Clustering methods for video browsing and annotation. In SPIE Storage and Retrieval for Image and Video Databases IV, pages 239-246,1996.
• 4: J. R. Kender , B. -L. Yeo, Video Scene Segmentation via Continuous Video Coherence, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.367, June 23-25, 1998
• 5: A.M. Ferman and A.M. Tekalp. Efficient filtering and clustering methods for temporal video segmentation and visual summarization. J. Vis. Commun. & lmage Rep., 9:336-351, 1998.
• 6: A.M. Ferman and A.M. Tekalp. Multiscale content extraction and representation for video indexing. In SPIE Multimedia Storage and Archiving Systems 11, 1997.
• 7: H. J. Zhang , C. Y. Low , S. W. Smoliar , J. H. Wu, Video parsing, retrieval and browsing: an integrated and content-based solution, Proceedings of the third ACM international conference on Multimedia, p.15-24, November 05-09, 1995, San Francisco, California, United States [doi>10.1145/217279.215068]
• 8: A. Hanjalic, M. Ceccarelli, R.L. Lagendijk, and J. Biemond. Automation of systems enabling search on stored video data. In SPIE Storage and Retrieval for Image and Video Databases V, pages 427--438,1997.
• 9: Daniel DeMenthon , Vikrant Kobla , David Doermann, Video summarization by curve simplification, Proceedings of the sixth ACM international conference on Multimedia, p.211-218, September 13-16, 1998, Bristol, United Kingdom [doi>10.1145/290747.290773]
• 10: Mark S. Drew , Jie Wei , Ze-Nian Li, Illumination-Invariant Color Object Recognition via Compressed Chromaticity Histograms of Color-Channel-Normalized Images, Proceedings of the Sixth International Conference on Computer Vision, p.533, January 04-07, 1998
• 11: J. Wei, M.S. Drew, and Z.N. Li. Illumination invariant video segmentation by hierarchical robust thresholding. In Electronic Imaging 198: Storage and Retrieval for Image and Video Databases I/1, pages 188-201. SPIE Vol. 3312, 1998.
• 12: G.D. Finlayson, P.M. Hubel, and S. Hordley. Colour by correlation. In Fifth Color Imaging Conf., pages 6-11, 1997.
• 13: E. Sahouria and A. Zakhor. Content analysis of video using principal components. 1EEE Trans. Circ. Sys. Vid. Tech., 9:1290-1298, 1999.
• 14: A. Girgensohnand J. Boreczky.Time-constrained key frame selection technique. In IEEE MM Sys., pages 756-761,1999.
• 15: M. S. Drew, J. Wei, and Z.N. Li. Illumination-invariant image retrieval and video segmentation. Pattern Recognition, 32:1369-1388, 1999.
• 16: C.E Borges. Trichromatic approximation method for surface illumination. J. Opt. Soc. Am. A, 8:1319-1323,1991.
• 17: Mark S. Drew, Ze-Nian Li., and Xiang Zhong. Video dissolve and wipe detection via spatio-temporal images of chromatic histogram differences. In 1CIP'O0, 2000. To appear.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Video (e.g., tape, disk, DVI)

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
  I.5 PATTERN RECOGNITION

General Terms:
Algorithms, Design, Measurement, Performance, Theory