Unsupervised soccer video abstraction based on pitch, dominant color and camera motion analysis

ABSTRACT

We present a soccer video abstraction method based on the analysis of the audio and video streams. This method could be applied to other sports as rugby or american football. The main contribution of this paper is the design of an unsupervised summarization method, and more specifically, the introduction of an efficient detector of excited speech segments. An excited commentary is supposed to correspond to an interesting moment of the game. It is simultaneously characterized by an increase of the pitch (or fundamental frequency) within the voiced segments and an increase of the energy supported by the harmonics of the pitch. The pitch is estimated from the autocorrelation function and its local increases are detected from a multiresolution technique. We introduce a specific energy measure for the voiced segments. A statistical analysis of the energy measures is performed to detect the most excited parts of the speech. A deterministic combination of excited speech detection, dominant color identification and camera motion analysis is then performed in order to discriminate between excited speech sequences of the game and excited speech sequences in commercials or in studio shots included in the processed TV programs.

The method presented here does not need any learning stage. It has been tested on seven soccer videos for a total duration of almost 20 hours.

REFERENCES

• 1: P. Bouthemy, M. Gelgon, and F. Ganansia. An unified approach to shot change detection and camera motion characterization. IEEE Trans. on Circuits and Systems for Video Technology, 9:1030--1044, 1999.
• 2: R. Cowie, E.Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J. Taylor. Emotion recognition in human-computer interaction. IEEE Signal Processing Magazine, pages 32--80, January 2001.
• 3: A. Ekin, A. M. Tekalp, and R. Mehrotra. Automatic soccer video analysis and summarization. IEEE Trans. on Image Processing, 12:796--807, July 2003.
• 4: D. Gerhard. Pitch extraction and fundamental frequency: history and current technique. Regina University of Canada, Nov.
• 5: A. Hanjalic. Generic approach to highlights extraction from a sport video. In ICIP'2003 Barcelona.
• 6: Leonardi, P. Migliorati, and M. Prandini. Semantic indexing of sports program sequences by audio-visual analysis. In ICIP'2003, Barcelona.
• 7: Surya Nepal , Uma Srinivasan , Graham Reynolds, Automatic detection of 'Goal' segments in basketball videos, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500181]
• 8: B. Li and I. Sezan. Event detection and summarization in sports video. In CVPR 2001, Hawai, pages Demos 19--21.
• 9: M. Petkovic, V. Mihajlovic, W. Jonker, and S. Djordjevic-Kajan. Multi-modal extraction of highlights from tv formula 1 programs. In ICME 2002, Ischia.
• 10: Y. Rui, A. Gupta, and A. Acero. Automatically extracting highligths for TV baseball programs. In ACM Multimedia 2003.
• 11: L. Xie, S. Chang, A. Divakaran, and H. Sun. Learning hierarchical hidden Markov models for video structure discovery. In Tech. Rep. 2002-006, ADVENT Group, Columbia Univ., Dec. 2002.
• 12: Lexing Xie , Peng Xu , Shih-Fu Chang , Ajay Divakaran , Huifang Sun, Structure analysis of soccer video with domain knowledge and hidden Markov models, Pattern Recognition Letters, v.25 n.7, p.767-775, May 2004 [doi>10.1016/j.patrec.2004.01.005]
• 13: D. Zhong and S. Chang. Structure analysis of sports video using domain models. In ICME'2001, Tokyo.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding

General Terms:
Algorithms

Keywords:
non verbal speech classification, video summarization