Live sports event detection based on broadcast video and web-casting text

ABSTRACT

Event detection is essential for sports video summarization, indexing and retrieval and extensive research efforts have been devoted to this area. However, the previous approaches are heavily relying on video content itself and require the whole video content for event detection. Due to the semantic gap between low-level features and high-level events, it is difficult to come up with a generic framework to achieve a high accuracy of event detection. In addition, the dynamic structures from different sports domains further complicate the analysis and impede the implementation of live event detection systems. In this paper, we present a novel approach for event detection from the live sports game using web-casting text and broadcast video. Web-casting text is a text broadcast source for sports game and can be live captured from the web. Incorporating web-casting text into sports video analysis significantly improves the event detection accuracy. Compared with previous approaches, the proposed approach is able to: (1) detect live event only based on the partial content captured from the web and TV; (2) extract detailed event semantics and detect exact event boundary, which are very difficult or impossible to be handled by previous approaches; and (3) create personalized summary related to certain event, player or team according to user's preference. We present the framework of our approach and details of text analysis, video analysis and text/video alignment. We conducted experiments on both live games and recorded games. The results are encouraging and comparable to the manually detected events. We also give scenarios to illustrate how to apply the proposed solution to professional and consumer services.

REFERENCES

• 1: Yong Rui , Anoop Gupta , Alex Acero, Automatically extracting highlights for TV Baseball programs, Proceedings of the eighth ACM international conference on Multimedia, p.105-115, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.354443]
• 2: Min Xu , N. C. Maddage , Changsheng Xu , M. Kankanhalli , Qi Tian, Creating audio keywords for event detection in soccer video, Proceedings of the 2003 International Conference on Multimedia and Expo, p.281-284, July 06-09, 2003
• 3: Chua Hock Chuan, Automatic Parsing of TV Soccer Programs, Proceedings of the International Conference on Multimedia Computing and Systems, p.167, May 15-18, 1995 [doi>10.1109/MMCS.1995.484921]
• 4: A. Ekin, A. M. Tekalp, and R. Mehrotra, "Automatic soccer video analysis and summarization", IEEE Trans. on Image Processing, vol. 12:7, no. 5, pp. 796--807, 2003.
• 5: Dongqing Zhang , Shih-Fu Chang, Event detection in baseball video using superimposed caption recognition, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641073]
• 6: Jürgen Assfalg , Marco Bertini , Carlo Colombo , Alberto Del Bimbo , Walter Nunziati, Semantic annotation of soccer videos: automatic highlights identification, Computer Vision and Image Understanding, v.92 n.2-3, p.285-305, November/December 2003 [doi>10.1016/j.cviu.2003.06.004]
• 7: R. Radhakrishan, Z. Xiong, A. Divakaran, Y. Ishikawa, "Generation of sports highlights using a combination of supervised & unsupervised learning in audio domain", In Proc. of International Conference on Pacific Rim Conference on Multimedia, Vol. 2, pp. 935--939, December 2003.
• 8: K. Wan, and C. Xu, "Robust soccer highlight generation with a novel dominant-speech feature extractor", In Proc. of IEEE International Conference on Multimedia and Expo, Taipei, Taiwan, pp.591--594, 27-30 Jun. 2004.
• 9: M. Xu, L. Duan, C. Xu, and Q. Tian, "A fusion scheme of visual and auditory modalities for event detection in sports video", In Proc. of IEEE International Conference on Acoustics, Speech, & Signal Processing, Hong Kong, China, Vol.3, pp.189--192, 2003.
• 10: Efficient Multimodal Features for Automatic Soccer Highlight Generation, Proceedings of the Pattern Recognition, 17th International Conference on (ICPR'04) Volume 3, p.973-976, August 23-26, 2004 [doi>10.1109/ICPR.2004.325]
• 11: M. Xu, L. Duan, C. Xu, M.S. Kankanhalli, and Q. Tian, "Event detection in basketball video using multi-modalities", In Proc. of IEEE Pacific Rim Conference on Multimedia, Singapore, Vol.3, pp.1526--1530, 15-18 Dec, 2003.
• 12: Mei Han , Wei Hua , Wei Xu , Yihong Gong, An integrated baseball digest system using maximum entropy method, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641081]
• 13: Surya Nepal , Uma Srinivasan , Graham Reynolds, Automatic detection of 'Goal' segments in basketball videos, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500181]
• 14: Jinjun Wang , Changsheng Xu , Engsiong Chng , Lingyu Duan , Kongwah Wan , Qi Tian, Automatic generation of personalized music sports video, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101309]
• 15: N. Nitta and N. Babaguchi, "Automatic story segmentation of closed-caption text for semantic content analysis of broadcasted sports video," In Proc. of 8th International Workshop on Multimedia Information Systems '02, pp. 110--116, 2002.
• 16: N. Babaguchi, Y. Kawai, and T. Kitahashi, "Event based indexing of broadcasted sports video by intermodal collaboration," IEEE Trans. on Multimedia, Vol. 4, pp. 68--75, March 2002.
• 17: Naoko Nitta , Noboru Babaguchi , Tadahiro Kitahashi, Generating Semantic Descriptions of Broadcasted Sports Videos Based on Structures of Sports Games and TV Programs, Multimedia Tools and Applications, v.25 n.1, p.59-83, January 2005 [doi>10.1023/B:MTAP.0000046382.62218.e1]
• 18: Huaxin Xu , Tat-Seng Chua, The fusion of audio-visual features and external knowledge for event detection in team sports video, Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval, October 15-16, 2004, New York, NY, USA [doi>10.1145/1026711.1026733]
• 19: Min Xu , N. C. Maddage , Changsheng Xu , M. Kankanhalli , Qi Tian, Creating audio keywords for event detection in soccer video, Proceedings of the 2003 International Conference on Multimedia and Expo, p.281-284, July 06-09, 2003
• 20: http://news.bbc.co.uk/sport2/hi/football/teams/
• 21: http://sports.espn.go.com/
• 22: M. Bertini , R. Cucchiara , A. Del Bimbo , A. Prati, Object and event detection for semantic annotation and transcoding, Proceedings of the 2003 International Conference on Multimedia and Expo, p.421-424, July 06-09, 2003
• 23: Riccardo Leonardi , Pierangelo Migliorati, Semantic Indexing of Multimedia Documents, IEEE MultiMedia, v.9 n.2, p.44-51, April 2002 [doi>10.1109/93.998057]
• 24: http://soccernet.espn.go.com/
• 25: Y. Tan and et al, "Rapid estimation of camera motion from compressed video with application to video annotation," IEEE Trans. on Circuits and Systems for Video Technology, vol. 10- 1, pp. 133--146, 2000.
• 26: Yiqun Li , Changsheng Xu , Kong Wah Wan , Xin Yan , Xinguo Yu, Reliable Video Clock Time Recognition, Proceedings of the 18th International Conference on Pattern Recognition, p.128-131, August 20-24, 2006 [doi>10.1109/ICPR.2006.993]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Abstracting methods

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

General Terms:
Algorithms, Experimentation, Measurement, Performance

Keywords:
broadcast video, event detection, web-casting text