Sports event detection using temporal patterns mining and web-casting text

ABSTRACT

Event detection is one of the essential tasks by which the performance of sports video content analysis and access becomes more efficient and effective. Among internal information which are extracted from inside raw videos, the temporal information is critical to convey event meaning. In this paper, the new method for adaptively detecting event based on Allen temporal algebra and external information support is presented. The temporal information is captured by presenting events as the temporal sequences using a lexicon of non-ambiguous temporal patterns. These sequences are then exploited to mine undiscovered sequences with external text information supports by using class associate rules mining technique. By modeling each pattern with "linguistic part" and "perceptual part" those work independently and connect together via "transformer", it is easy to deploy this method to any new domain (e.g baseball, basketball, tennis, etc.) with a few changes in "perceptual part" and "transformer". Thus the proposed method not only can work well in unwell structured environments but also can be able to adapt itself to new domains without the need (or with a few modification) for external re-programming, re-configuring and re-adjusting. Experimental results that are carried on more than 30 hours of soccer video corpus captured at different broadcasters and conditions as well as compared with well-known related methods, demonstrated the efficiency, effectiveness, and robustness of the proposed method in both offline and online processes.

REFERENCES

• 1: James F. Allen, Maintaining knowledge about temporal intervals, Communications of the ACM, v.26 n.11, p.832-843, Nov. 1983 [doi>10.1145/182.358434]
• 2: J. Calic, N. Campbell, S. Dasiopoulou, and Y. Kompatsiaris. An overview of multimodal video representation for semantic analysis. In European Workshop on the Integration of Knowledge, Semantics and Digital Media Technologies, Conference Proceedings, pages 39--45, December 2005.
• 3: M. Chen, S. Chen, and M. Shyu. Hierarchical temporal association mining for video event detection in video databases. In MDDM'06 Conference Proceedings, pages 137--145. IEEE, April 2007.
• 4: M. Chen, S. Chen, M. Shyu, and K. Wickramaratna. Semantic event detection via multimodal data mining. IEEE Signal Processing Magazine, pages 38--46, March 2006.
• 5: L. Duan, M. Xu, Q. Tian, C. Xu, and J. Jin. A unified framework for semantic shot representation of sports video. IEEE Trans. on Multimedia, 7(6):1066--1083, December 2005.
• 6: A. Ekin, A. Tekalp, and R. Mehrotta. Automatic soccervideo analysis and summarization. IEEE Trans. on Image Processing, 12(7):796--807, July 2003.
• 7: Michael Fleischman , Phillip Decamp , Deb Roy, Mining temporal patterns of movement for video content classification, Proceedings of the 8th ACM international workshop on Multimedia information retrieval, October 26-27, 2006, Santa Barbara, California, USA [doi>10.1145/1178677.1178704]
• 8: Michael Fleischman , Deb Roy, Unsupervised content-based indexing of sports video, Proceedings of the international workshop on Workshop on multimedia information retrieval, September 24-29, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290082.1290097]
• 9: S. Jiang, Q. Huang, and W. Gao. Effective image and video mining: an overview of model-based approaches. In ICME'07 Conference Proceedings, pages 1095--1098. IEEE, July 2007.
• 10: Y. Liu, S. Jiang, Q. Ye, W. Gao, and Q. Huang. Playfield detection using adaptive gmm and its application. In ICASSP'05 Conference Proceeding, pages 421--424. ACM, March 2005.
• 11: Rokia Missaoui , Roman M. Palenichka, Effective image and video mining: an overview of model-based approaches, Proceedings of the 6th international workshop on Multimedia data mining: mining integrated media and complex data, p.43-52, August 21-21, 2005, Chicago, Illinois [doi>10.1145/1133890.1133895]
• 12: Jian Pei , Jiawei Han , Behzad Mortazavi-Asl , Jianyong Wang , Helen Pinto , Qiming Chen , Umeshwar Dayal , Mei-Chun Hsu, Mining Sequential Patterns by Pattern-Growth: The PrefixSpan Approach, IEEE Transactions on Knowledge and Data Engineering, v.16 n.11, p.1424-1440, November 2004 [doi>10.1109/TKDE.2004.77]
• 13: D. Sadlier and N. O'Connor. Event detection in field sports video using audio-visual features and a support vector machine. IEEE Trans. on Circuits and Systems for Video Technology, 15(10):1225--1233, October 2005.
• 14: C. Snoek and M. Worring. Multimedia eventbased video indexing using time intervals. IEEE Trans. on Multimedia, 7(4):638--647, August 2005.
• 15: Cees G. M. Snoek , Marcel Worring, Multimodal Video Indexing: A Review of the State-of-the-art, Multimedia Tools and Applications, v.25 n.1, p.5-35, January 2005 [doi>10.1023/B:MTAP.0000046380.27575.a5]
• 16: Xiaofeng Tong , Hanqing Lu , Qingshan Liu , Hongliang Jin, Replay Detection in Broadcasting Sports Video, Proceedings of the Third International Conference on Image and Graphics, p.337-340, December 18-20, 2004 [doi>10.1109/ICIG.2004.124]
• 17: F. Wang, L. Sun, B. Yang, and S. Yang. Fast arc detection algorithm for playfield registration in soccer video mining. In Systems, Man, and Cybernetics Conference Proceedings, pages 4932--4936. IEEE, October 2006.
• 18: Shin-Yi Wu , Yen-Liang Chen, Mining Nonambiguous Temporal Patterns for Interval-Based Events, IEEE Transactions on Knowledge and Data Engineering, v.19 n.6, p.742-758, June 2007 [doi>10.1109/TKDE.2007.190613]
• 19: Z. Xiong, X. Zhou, Q. Tian, Y. Rui, and T. Huang. Semantic retrieval of video. IEEE Signal Processing Magazine, pages 18--27, March 2006.
• 20: Changsheng Xu , Jinjun Wang , Kongwah Wan , Yiqun Li , Lingyu Duan, Live sports event detection based on broadcast video and web-casting text, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180699]
• 21: Q. Zhao and S. Bhowmick. Sequential pattern mining: A survey. Itechnical Report CAIS Nayang Technological University Singapore, pages 1--26, 2003.
• 22: Xingquan Zhu , Xindong Wu , Ahmed K. Elmagarmid , Zhe Feng , Lide Wu, Video Data Mining: Semantic Indexing and Event Detection from the Association Perspective, IEEE Transactions on Knowledge and Data Engineering, v.17 n.5, p.665-677, May 2005 [doi>10.1109/TKDE.2005.83]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Abstracting methods

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

General Terms:
Algorithms, Experimentation, Measurement, Performance

Keywords:
data-mining, event detection, temporal sequential patterns, web-casting text