An instant semantics acquisition system of live soccer video with application to live event alert and on-the-fly language selection

ABSTRACT

Automatic indexing of recorded sports video is a hard problem, even more so for live sports video, though a lot of advances have been achieved in the recent years. This paper presents a semi-automatic instant semantics generation system of live soccer video and its two applications: live event alert and instant semantics embedding. A key technical challenge of building the proposed system is how to acquire the desired semantics in a very short time lag. To confront this challenge, we develop a three-channel approach to accurately and quickly acquire semantics. The hree channels are (1) developing the tool for inputting gamelog, (2) signal communication for getting commands of referee and director, and (3) video analysis for finding event boundary. The proposed system requires only a very small amount of manual work to operate gamelog acquisition tool using an icon-based interface for entering event log. Experimental results show that our system can quickly obtain the semantics of live soccer video, alert live events with a very small latency, and embed semantics into broadcasting video with a short time lag.

REFERENCES

• 1: Jürgen Assfalg , Marco Bertini , Carlo Colombo , Alberto Del Bimbo , Walter Nunziati, Semantic annotation of soccer videos: automatic highlights identification, Computer Vision and Image Understanding, v.92 n.2-3, p.285-305, November/December 2003 [doi>10.1016/j.cviu.2003.06.004]
• 2: Babaguchi N., Kawai Y., and Kitashi T., Event based semantics of broadcasted sports video by intermodal collaboration, IEEE Trans. on Multimedia, vol. 4, pp. 68--75, March, 2002.
• 3: R. M. Bolle , B.-L. Yeo , M. M. Yeung, Video query: research directions, IBM Journal of Research and Development, v.42 n.2, p.233-252, March 1998
• 4: Brunelli R., Mich O., and Modena C.M., A survey of the automatic semantics of video data, J. of Vis. Com. & Image Representation 10(2), 78--112, 1999.
• 5: IEEE MultiMedia, The Holy Grail of Content-Based Media Analysis, IEEE MultiMedia, v.9 n.2, p.6-10, April 2002 [doi>10.1109/93.998041]
• 6: Chen S.-C, Shyu M.-L., Chen M, and Zhang C., A decision tree-based multimodal data mining framework for soccer goal detection, ICME 2004, pp. 265--268, Taibei, Taiwan, June, 2004.
• 7: Ling-Yu Duan , Min Xu , Tat-Seng Chua , Qi Tian , Chang-Sheng Xu, A mid-level representation framework for semantic sports video analysis, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957020]
• 8: Ekin A., Tekalp A. M., and Mehrotra R., Automatic soccer video analysis and summarization, IEEE Trans. Image Processing, vol. 12, no. 7, pp. 796--807, 2003.
• 9: Chua Hock Chuan, Automatic Parsing of TV Soccer Programs, Proceedings of the International Conference on Multimedia Computing and Systems, p.167, May 15-18, 1995 [doi>10.1109/MMCS.1995.484921]
• 10: Leonardi R., Migliorati P., and Prandini M., Semantics acquisition of soccer audio-visual sequences: a multimodal approach based on controlled Markov chains, CVST, vol. 14, 2004, pp: 634 -- 643.
• 11: Li B., Errico J., Pan H., and Sezan M. I., Bridging the semantic gap in sports video retrieval and summarization, J. of Vis. Commun. & Image Representation, vol 15, 2004, pp. 393--424.
• 12: Ngo C. W., Zhang H. J., and Pong T. C., Recent advances in content-based video analysis, J. of Image and Graphics, vol. 3, no. 1, pp.445--468, 2001.
• 13: Pan H., Li B., and Sezan M. I., Automatic detection of replay segments in broadcast sports programs by detection of logos in scene transitions, ICASSP 2002.
• 14: H. Pan , P. van Beek , M. I. Sezan, Detection of slow-motion replay segments in sports video for highlights generation, Proceedings of the Acoustics, Speech, and Signal Processing, 2001. on IEEE International Conference, p.1649-1652, May 07-11, 2001 [doi>10.1109/ICASSP.2001.941253]
• 15: Pingali G. S., Opalach A., Jean Y. D., and Carlbom I. B., Instantly indexed multimedia databases of real world events, IEEE Trans. Multimedia, vol. 4, pp. 269--282, June 2002.
• 16: Cees G. M. Snoek , Marcel Worring, Multimodal Video Indexing: A Review of the State-of-the-art, Multimedia Tools and Applications, v.25 n.1, p.5-35, January 2005 [doi>10.1023/B:MTAP.0000046380.27575.a5]
• 17: Tovinkere V. and Qian R. J., Detecting semantic events in soccer games: Towards a complete solution, ICME 2001, 1040--1043.
• 18: Wan K., Yan X., and Xu C., Automatic mobile sports highlights, ICME 2005.
• 19: Jinjun Wang , Changsheng Xu , Engsiong Chng , Kongwah Wah , Qi Tian, Automatic replay generation for soccer video broadcasting, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027535]
• 20: Wang Y., Liu Z., and Huang J., Multimedia content analysis using both audio and visual cues, IEEE Signal Proc. Magazine, vol. 17, pp.12--36, 2000.
• 21: Lexing Xie , Peng Xu , Shih-Fu Chang , Ajay Divakaran , Huifang Sun, Structure analysis of soccer video with domain knowledge and hidden Markov models, Pattern Recognition Letters, v.25 n.7, p.767-775, May 2004 [doi>10.1016/j.patrec.2004.01.005]
• 22: Changsheng Xu , Jinjun Wang , Kongwah Wan , Yiqun Li , Lingyu Duan, Live sports event detection based on broadcast video and web-casting text, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180699]
• 23: Huaxin Xu , Tat-Seng Chua, The fusion of audio-visual features and external knowledge for event detection in team sports video, Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval, October 15-16, 2004, New York, NY, USA [doi>10.1145/1026711.1026733]
• 24: Huaxin Xu , Tat-Seng Chua, Fusion of AV features and external information sources for event detection in team sports video, ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), v.2 n.1, p.44-67, February 2006 [doi>10.1145/1126004.1126007]
• 25: Min Xu , Ling-Yu Duan , Chang-Sheng Xu , Qi Tian, A fusion scheme of visual and auditory modalities for event detection in sports video, Proceedings of the 2003 International Conference on Multimedia and Expo, p.333-336, July 06-09, 2003
• 26: Xiong Z., Radhakrishnan R., and Divakaran A., Generation of sports highlights using motion activity in combination with a common audio feature extraction framework, ICIP 2003, vol. 1, pp. 5--8, 2003.
• 27: Yang Y.-Q., Lu Y-D, and Chen W., A framework for automatic detection of soccer goal event based on cinematic template, Int'l Conf. on Machine Learning and Cybernetics, vol 6, pp.3759--764, 2004.
• 28: Yow D., Yeo B. L., Yeung M., and Liu B., Analysis and presentation of soccer highlight from digital video, ACCV 1995, pp.499--503, 1995.
• 29: http://news.bbc.co.uk/sport1/hi/football/teams/m/man_utd/live_text/default.stm
• 30: http://live.espnstar.com.cn/tools/live/newlivelist.asp
• 31: http://www.microsoft.com/tv/IPTVEdition.mspx

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.5 On-line Information Services
          Subjects: Web-based services

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems

General Terms:
Algorithms, Experimentation, Management

Keywords:
Gamelog, icon-based input, instant semantics acquisition, live event alert, on-the-fly language selection, sports video