THU-intel at rushes summarization of TRECVID 2008

ABSTRACT

Video summary is an active research field to help users to grasp a whole video's content for efficient browsing and editing. In this paper, we describe our THU-Intel rushes summarization system in TRECVID2008. In our approach, we first extract low-level audiovisual features and parse the video into shots, sub-shots and 1-second video clips. Then we remove junk video clips with color-bar, near uniform-color and clapboard frames etc. To select video clips with main objects and events, we evaluate each clip's representative score by multimodal features of color, edge, motion, and audio etc. Finally, we construct the rushes video summary by iteratively selecting the most representative video clips and removing similar ones. Extensive experiments are carried out on 40 testing rushes videos. Good results demonstrate the effectiveness of the proposed method.

REFERENCES

• 1: Alexander G. Hauptmann , Michael G. Christel , Wei-Hao Lin , Bryan Maher , Jun Yang , Robert V. Baron , Guang Xiang, Clever clustering vs. simple speed-up for summarizing rushes, Proceedings of the international workshop on TRECVID video summarization, p.20-24, September 28-28, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290031.1290034]
• 2: Jim Kleban , Anindya Sarkar , Emily Moxley , Stephen Mangiat , Swapna Joshi , Thomas Kuo , B. S. Manjunath, Feature fusion and redundancy pruning for rush video summarization, Proceedings of the international workshop on TRECVID video summarization, p.84-88, September 28-28, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290031.1290047]
• 3: Ba Tu Truong , Svetha Venkatesh, Generating comprehensible summaries of rushes sequences based on robust feature matching, Proceedings of the international workshop on TRECVID video summarization, p.30-34, September 28-28, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290031.1290036]
• 4: Chen-Ming Pan , Yung-Yu Chuang , Winston H. Hsu, NTU TRECVID-2007 fast rushes summarization system, Proceedings of the international workshop on TRECVID video summarization, p.74-78, September 28-28, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290031.1290045]
• 5: Cong Li, Zhijian Ou, Wei hu, Tao Wang, Yimin Zhang, Caption-aided speech detection in videos, ICASSP 2008, pp141--144.
• 6: David G. Lowe, Object Recognition from Local Scale-Invariant Features, Proceedings of the International Conference on Computer Vision-Volume 2, p.1150, September 20-25, 1999
• 7: Feng Wang , Chong-Wah Ngo, Rushes video summarization by object and event understanding, Proceedings of the international workshop on TRECVID video summarization, p.25-29, September 28-28, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290031.1290035]
• 8: Li, Z., Schuster, G. M., Katsaggelos, A. K., Rate-Distortion Optimial Video Summary Generation, IEEE Trans. on Image Processing, 14(10), 2005.
• 9: Martin A. Fischler , Robert C. Bolles, Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography, Communications of the ACM, v.24 n.6, p.381-395, June 1981 [doi>10.1145/358669.358692]
• 10: Navneet Dalal , Bill Triggs, Histograms of Oriented Gradients for Human Detection, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1, p.886-893, June 20-26, 2005 [doi>10.1109/CVPR.2005.177]
• 11: P. Over, P., Smeaton, A.F. and Kelly, P., The TRECVID 2007 BBC Rushes Summarization Evaluation Pilot. In Proc. of the TRECVID Workshop on Video Summarization (TVS'07), ACM Multimedia, 2007.
• 12: Park, J. I., Yagi, N., Enami, K., Aizawa, K., Hatori, M., Estimation of camera parameters from image sequence for model-based video coding, CirSysVideo(4), No. 3, June 1994, pp. 288--296.
• 13: Paul Over , Alan F. Smeaton , George Awad, The trecvid 2008 BBC rushes summarization evaluation, Proceedings of the 2nd ACM TRECVid Video Summarization Workshop, p.1-20, October 31-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1463563.1463564]
• 14: Tao Wang , Yue Gao , Jianguo Li , Patricia P. Wang , Xiaofeng Tong , Wei Hu , Yimin Zhang , Jianmin Li, THU-ICRC at rush summarization of TRECVID 2007, Proceedings of the international workshop on TRECVID video summarization, p.79-83, September 28-28, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290031.1290046]
• 15: Shingo Uchihashi , Jonathan Foote , Andreas Girgensohn , John Boreczky, Video Manga: generating semantically meaningful video summaries, Proceedings of the seventh ACM international conference on Multimedia (Part 1), p.383-392, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319463.319654]
• 16: Werner Bailer , Felix Lee , Georg Thallinger, Skimming rushes video using retake detection, Proceedings of the international workshop on TRECVID video summarization, p.60-64, September 28-28, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290031.1290042]
• 17: Yuan, J. H., Zheng, W. J., Chen, L., etc. Tsinghua University at TRECVID 2004: shot boundary detection and high-level feature extraction, In NIST workshop of TRECVID 2004.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL

General Terms:
Algorithms, Design, Experimentation, Management

Keywords:
camera motion estimation, clapboard detection, junk clip removing, ransac, representative score, video summarization