Rushes video summarization using audio-visual information and sequence alignment

ABSTRACT

This paper describes our system and methodologies for the BBC rushes video summarization task of TRECVID 2008. The procedure of the system is composed of three major steps: shot detection, irrelevant and repetitive subshot removal, and final summary generation. First, we segment the original rushes video into subshots according to the difference and accumulative difference of local color histogram between consecutive frames. Second, we recognize the irrelevant subshots, such as subshots of color bar, pure gray frames, and clapper board. We propose a novel video sequence alignment algorithm to detect repetitive subshots. After removing the irrelevant and repetitive subshots, we generate the final summary using the remaining informative and representative subshots. The evaluation from TRECVID 2008 shows that our system can generate good video summaries.

REFERENCES

• 1: Takao, S., Haru, T., and Ariki, Y. Summarization of News Speech with Unknown Topic Boundary. In Proceedings of the IEEE International Conference on Multimedia and Expo. pp: 615--618, 2001.
• 2: Ekin, A., Tekalp, A. M., and Mehrotra, R. Automatic Soccer Video Analysis and Summarization. IEEE Transactions on Image Processing. vol. 12, no. 7, pp. 796--807, July 2003.
• 3: Choudary, C. and Liu. T.-C. Summarization of Visual Content in Instructional Videos. IEEE Transactions on Multimedia. vol. 9, no. 7, pp. 1443--1455, Nov. 2007.
• 4: Paul Over , Alan F. Smeaton , George Awad, The trecvid 2008 BBC rushes summarization evaluation, Proceedings of the 2nd ACM TRECVid Video Summarization Workshop, p.1-20, October 31-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1463563.1463564]
• 5: HongJiang Zhang , Atreyi Kankanhalli , Stephen W. Smoliar, Automatic partitioning of full-motion video, Multimedia Systems, v.1 n.1, p.10-28, June 1993 [doi>10.1007/BF01210504]
• 6: Yang Liu , Yan Liu , Yan Zhang, The Hong Kong Polytechnic University at TRECVID 2007 BBC rushes summarization, Proceedings of the international workshop on TRECVID video summarization, p.50-54, September 28-28, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290031.1290040]
• 7: Pfeiffer, S., Lienhart, R., Fischer, S., and Effelsberg, W. Abstracting Digital Movies Automatically. Journal of Visual Communication and Image Representation, 7, 4, pp: 345--353. Dec. 1996.
• 8: Yan, J. X, Dou, W. B., and Dong, Z. W. Time-Domain Detection Method of Transient Signals in Audio Coding. Journal of Electronics & Information Technology, vol. 28, no. 2, Feb. 2006.
• 9: Chang, C. C. and Lin, C. J. LIBSVM: a library for support vector machines, 2001.
• 10: Manjunath, B. S., Ohm, J. R., Vasudevan, V. V., and Yamada, A. Color and Texture Descriptors. IEEE Transactions on Circuits and Systems for Video Technology, vol. 11, pp. 703--715, 2001.
• 11: Needleman, S. and Wunsch, C. A General Method Applicable to the Search for Similarities in the Amino Acid Sequence of Two Proteins. Journal of Molecular Biology, vol. 48, no. 3, pp. 443--453, 1970.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Video analysis

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Abstracting methods

General Terms:
Algorithms, Experimentation

Keywords:
rushes editing, trecvid, video summarization