Video summarization preserving dynamic content

ABSTRACT

This paper describes a system for selecting excerpts from unedited video and presenting the excerpts in a short summary video for efficiently understanding the video contents. Color and motion features are used to divide the video into segments where the color distribution and camera motion are similar. Segments with and without camera motion are clustered separately to identify redundant video. Audio features are used to identify clapboard appearances for exclusion. Representative segments from each cluster are selected for presentation. To increase the original material contained within the summary and reduce the time required to view the summary, selected segments are played back at a higher rate based on the amount of detected camera motion in the segment. Pitch-preserving audio processing is used to better capture the sense of the original audio. Metadata about each segment is overlayed on the summary to help the viewer understand the context of the summary segments in the original video.

REFERENCES

• 1: J.-Y. Bouget. Pyramidal implementation of the lucas kanade feature tracker. description of the algorithm. Technical report, Intel Corporation Microprocessor Research Lab, 2000.
• 2: M. Cooper and J. Foote. Scene boundary detection via video self-similarity analysis. In Proc. IEEE International Conference on Image Processing (ICIP), pages 378--81, 2001.
• 3: B. Gunsel, M. Ferman, and A. M. Tekalp. Temporal video segmentation using unsupervised clustering and semantic object tracking. Journal of Electronic Imaging, 7:592--604, July 1998.
• 4: Thomas Hofmann, Unsupervised learning by probabilistic latent semantic analysis, Machine Learning, v.42 n.1/2, p.177-196, Jan. 2001 [doi>10.1023/A:1007617005950]
• 5: B. Lucas and T. Kanade. An iterative image registration technique with an application to stereo vision. In IJCAI81, pages 674--679, 1981.
• 6: A. Ng, M. Jordan, and Y. Weiss. On spectral clustering: Analysis and an algorithm. In Advances in Neural Information Processing Systems 14: Proceedings of the 2001., 2001.
• 7: J.-M. Odobez, D. Gatica-Perez, and M. Guillemot. Video shot clustering using spectral methods. In Int. Workshop on Content-based Multimedia Indexing (CBMI), 2003.
• 8: Open source computer vision library. http://www.intel.com/technology/computing/opencv/.
• 9: Paul Over , Alan F. Smeaton , Philip Kelly, The trecvid 2007 BBC rushes summarization evaluation pilot, Proceedings of the international workshop on TRECVID video summarization, p.1-15, September 28-28, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290031.1290032]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Abstracting methods

General Terms:
Algorithms, Experimentation, Performance

Keywords:
clustering, presentation, segmentation, video summarization