Automatic replay generation for soccer video broadcasting

ABSTRACT

While most current approaches for sports video analysis are based on broadcast video, in this paper, we present a novel approach for highlight detection and automatic replay generation for soccer videos taken by the main camera. This research is important as current soccer highlight detection and replay generation from a live game is a labor-intensive process. A robust multi-level, multi-model event detection framework is proposed to detect the event and event boundaries from the video taken by the main camera. This framework explores the possible analysis cues, using a mid-level representation to bridge the gap between low-level features and high-level events. The event detection results and mid-level representation are used to generate replays which are automatically inserted into the video. Experimental results are promising and found to be comparable with those generated by broadcast professionals.

REFERENCES

• 1: N. Adami, R. Leonardi, and P. Migliorati, "An overview of multi-modal techniques for the characterization of sport programmes," SPIE-VCIP'03, pp. 1296--1306, 2003.
• 2: Xinguo Yu , Changsheng Xu , Hon Wai Leong , Qi Tian , Qing Tang , Kong Wah Wan, Trajectory-based ball detection and tracking with applications to semantic analysis of broadcast soccer video, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957018]
• 3: Ling-Yu Duan , Min Xu , Tat-Seng Chua , Qi Tian , Chang-Sheng Xu, A mid-level representation framework for semantic sports video analysis, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957020]
• 4: Jürgen Assfalg , Marco Bertini , Carlo Colombo , Alberto Del Bimbo , Walter Nunziati, Semantic annotation of soccer videos: automatic highlights identification, Computer Vision and Image Understanding, v.92 n.2-3, p.285-305, November/December 2003 [doi>10.1016/j.cviu.2003.06.004]
• 5: A. Ekin, A. M. Tekalp, and R. Mehrotra, "Automatic soccer video analysis and summarization," IEEE Trans. on Image Processing, vol. 12:7, pp. 796--807, 2003.
• 6: N. Babaguchi and N. Nitta, "Intermodal collaboration: A strategy for semantic content analysis for broadcasted sports video," ICIP'03, vol. 1, pp. 13--16, 2003.
• 7: Z. Xiong., R. Radhakrishnan, and A. Divakaran, "Generation of sports highlights using motion activity in combination with a common audio feature extraction framework," ICIP'03, vol. 1, pp. 5--8, 2003.
• 8: N. Nitta and N. Babaguchi, "Automatic story segmentation of closed-caption text for semantic content analysis of broadcasted sports video," Inter. Workshop on MM Info. Sys. '02, pp. 110--116, 2002.
• 9: Dongqing Zhang , Shih-Fu Chang, Event detection in baseball video using superimposed caption recognition, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641073]
• 10: Y. Ma and H. Zhang, "Motion pattern based video classification using support vector machines," ISCAS'02, Theme: Circuits and Systems for Ubiquitous Computing, 2002.
• 11: Y. Tan and et al, "Rapid estimation of camera motion from compressed video with application to video annotation," IEEE Trans. on Circuits and Systems for Video Technology, vol. 10-1, pp. 133--146, 2000.
• 12: Lexing Xie , Peng Xu , Shih-Fu Chang , Ajay Divakaran , Huifang Sun, Structure analysis of soccer video with domain knowledge and hidden Markov models, Pattern Recognition Letters, v.25 n.7, p.767-775, May 2004 [doi>10.1016/j.patrec.2004.01.005]
• 13: J. Wang, C. Xu, E. Chng, and Q. Tian, "Sports highlight detection from keyword sequences using hmm," ICME'04, 2004.
• 14: I. F. A. Board, "Law of the game," Federation International de Football Association, 11 hitzigweg, 8030 Zurich, Switzerland, July 2003.
• 15: M. Xu and et al, "Creating audio keywords for event detection in soccer video," ICME'03, vol. 2, pp. 281--284, 2003.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.5 Implementation
          Subjects: Interactive systems

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods; Abstracting methods

General Terms:
Algorithms, Design, Experimentation

Keywords:
broadcast, event detection, replay, sports video analysis