Automatic generation of personalized music sports video

ABSTRACT

In this paper, we propose a novel automatic approach for personalized music sports video generation. Two research challenges, semantic sports video content selection and automatic video composition, are addressed. For the first challenge, we propose to use multi-modal (audio, video and text) feature analysis and alignment to detect the semantic of events in sports video. For the second challenge, we propose video-centric and music-centric music video composition schemes to automatically generate personalized music sports video based on user's preference. The experimental results and user evaluations are promising and show that our system's generated music sports video is comparable to manually generated ones. The proposed approach greatly facilitates the automatic music sports video generation for both professionals and amateurs.

REFERENCES

• 1: MuVee Technologies Pte. Ltd, "Muvee TM," 2000.
• 2: N. Adami, R. Leonardi, and P. Migliorati, "An overview of multi-modal techniques for the characterization of sport programmes," Proc. of SPIE-VCIP'03, pp. 1296--1306, July, 2003.
• 3: J. Wang, E. Chng, and C. Xu, "Soccer replay detection using scene transition structure analysis," Proc. of IEEE ICASSP'05, March 2005.
• 4: J. Wang, et al, "Event detection based on non-broadcast sports video," Proc. of IEEE ICIP'04, Nov. 2004.
• 5: A. Ekin, A. Tekalp, and R. Mehrotra, "Automatic soccer video analysis and summarization," IEEE Trans. on Image Processing, vol. 12:7, no. 5, pp. 796--807, 2003.
• 6: Jürgen Assfalg , Marco Bertini , Carlo Colombo , Alberto Del Bimbo , Walter Nunziati, Semantic annotation of soccer videos: automatic highlights identification, Computer Vision and Image Understanding, v.92 n.2-3, p.285-305, November/December 2003 [doi>10.1016/j.cviu.2003.06.004]
• 7: N. Babaguchi and N. Nitta, "Intermodal collaboration: A strategy for semantic content analysis for broadcasted sports video," Proc. of IEEE ICIP'03, vol. 1, pp. 13--16, Sept. 2003.
• 8: Xian-Sheng HUA , Lie LU , Hong-Jiang ZHANG, Automatic music video generation based on temporal pattern analysis, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027641]
• 9: Jonathan Foote , Matthew Cooper , Andreas Girgensohn, Creating music videos using automatic media analysis, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641119]
• 10: Huaxin Xu , Tat-Seng Chua, The fusion of audio-visual features and external knowledge for event detection in team sports video, Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval, October 15-16, 2004, New York, NY, USA [doi>10.1145/1026711.1026733]
• 11: MediaWare Solutions Pte. Ltd (USA), "M2-edit pro TM," 2002.
• 12: H. Pan, B. Li, and M. Sezan, "Automatic detection of replay segments in broadcast sports programs by detection of logos in scene transitions," Proc. of IEEE ICASSP'02, May 2002.
• 13: V. Kobla, D. DeMenthon, and D. Doermann, "Detection of slow-motion replay sequences for identifying sports videos," Proc. IEEE Workshop on Multimedia Signal Processing, 1999.
• 14: H. Pan, B. Li, and M. Sezan, "Detection of slow-motion replay segments in sports video for highlights generation," Proc. of IEEE ICASSP'01, May 2001.
• 15: Jinjun Wang , Changsheng Xu , Engsiong Chng , Kongwah Wah , Qi Tian, Automatic replay generation for soccer video broadcasting, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027535]
• 16: N. Babaguchi, Y. Kawai, and T. Kitahashi, "Event based indexing of broadcasted sports video by intermodal collaboration," IEEE Trans. on Multimedia, vol. 4, pp. 68--75, March 2002.
• 17: Naoko Nitta , Noboru Babaguchi , Tadahiro Kitahashi, Generating Semantic Descriptions of Broadcasted Sports Videos Based on Structures of Sports Games and TV Programs, Multimedia Tools and Applications, v.25 n.1, p.59-83, January 2005 [doi>10.1023/B:MTAP.0000046382.62218.e1]
• 18: N. Nitta and N. Babaguchi, "Automatic story segmentation of closed-caption text for semantic content analysis of broadcasted sports video," Proc. of 8th International Workshop on MIS'02, pp. 110--116, 2002.
• 19: "http://news.bbc.co.uk/sport1/hi/football/teams/,"
• 20: Christopher D. Manning , Hinrich Schütze, Foundations of statistical natural language processing, MIT Press, Cambridge, MA, 1999
• 21: dtSearch Corp, "dtsearch 6.50 (6608)," 1991-2005.
• 22: "Hidden markov model toolkit," http://htk.eng.cam.ac.uk/.
• 23: J. Pylkkonen and M. Kurimo, "Duration modeling techniques for continuous speech recognition," Proc. of IEEE ICASSP'04, pp. 385--388, May 2004.
• 24: Namunu C. Maddage , Changsheng Xu , Mohan S. Kankanhalli , Xi Shao, Content-based music structure analysis with applications to music semantics understanding, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027549]
• 25: John P. Chin , Virginia A. Diehl , Kent L. Norman, Development of an instrument measuring user satisfaction of the human-computer interface, Proceedings of the SIGCHI conference on Human factors in computing systems, p.213-218, May 15-19, 1988, Washington, D.C., United States [doi>10.1145/57167.57203]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.5 Implementation
          Subjects: Interactive systems

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods; Abstracting methods

General Terms:
Algorithms, Design, Experimentation

Keywords:
automatic video editing, event detection, personalized music sports video, sports video analysis, video content selection