Hierarchical modeling and adaptive clustering for real-time summarization of rush videos in trecvid'08

ABSTRACT

In this paper, our techniques used in TRECVID'08 on BBC rush summarization are described. Firstly, rush videos are hierarchical modeled using formal language description. Then, shot detection and V-unit determination are applied for video structuring; junk frames within the model are also effectively removed. Thirdly, adaptive clustering is employed to group shots into clusters to remove retakes. Then, each selected shot is ranked according to its length and sum of activity level for summarization. Competitive results have proved the effectiveness and efficiency of our techniques fully implemented in compressed-domain.

REFERENCES

• 1: Xu, L.-Q., Hanjalic, A. 2005. Affective video content representation and modelling. IEEE T-Multimedia. 7,1 (Feb. 2005), 143--154.
• 2: Ferman, A. M., and Tekalp, A. M. 2003. Two-stage hierarchical video summary extraction to match low-level user browsing preferences. IEEE T--Multimedia. 5, 2 (2003), 244--256.
• 3: Zhu, X., Elmagarmid, A. K., Xue, X., Wu, L., and Catlin, A. C. 2005. InsightVideo: toward hierarchical video content organization for efficient browsing, summarization and retrieval. IEEE T-Multimedia. 7,4 (2005), 648--666.
• 4: Guironnet, M., Pellerin, D., Guyader, N., and Ladret, P. 2007. Video summarization based on camera motion and a subjective evaluation model. EURASIP J. Image and Video Processing. Article ID 60245.
• 5: Yu-Fei Ma , Lie Lu , Hong-Jiang Zhang , Mingjing Li, A user attention model for video summarization, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641116]
• 6: Arthur G. Money , Harry Agius, Video summarisation: A conceptual framework and survey of the state of the art, Journal of Visual Communication and Image Representation, v.19 n.2, p.121-143, February, 2008 [doi>10.1016/j.jvcir.2007.04.002]
• 7: Chong-Wah Ngo , Yu-Fei Ma , Hong-Jiang Zhang, Automatic Video Summarization by Graph Modeling, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.104, October 13-16, 2003
• 8: Ray, S., and Turi, R. H. 1999. Determination of number clusters in K-means clustering and application in colour image segmentation. In Proc. the 4th Int. Conf. Advances in Pattern Recognition and Digital techniques (Dec. 27--29, 1999), Calcutta, India, 137--143.
• 9: Paul Over , Alan F. Smeaton , George Awad, The trecvid 2008 BBC rushes summarization evaluation, Proceedings of the 2nd ACM TRECVid Video Summarization Workshop, p.1-20, October 31-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1463563.1463564]
• 10: Ren, J.-C., Jiang, J. M., and Chen, J. 2007. Determination of shot boundary in MPEG videos for TRECVID 2007. In TREC Video Retrieval Evaluation Online Proceedings, http://www-nlpir.nist.gov/ projects/tvpubs/tv7.papers/bradford.pdf.
• 11: Shi L., King, I., Lyu, M.R. 2004. Video summarization by video structure analysis and graph optimization. In Proceedings Int. Conf. Multimedia and Expo (June 27--30, 2004), Taipei, R.O.C., 1959--1962.
• 12: Ba Tu Truong , Svetha Venkatesh, Video abstraction: A systematic review and classification, ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), v.3 n.1, p.3-es, February 2007 [doi>10.1145/1198302.1198305]
• 13: Li, Y., Lee, S.-H., yeh, C.-H., and Kuo, C.-C. J. 2006. Techniques for movie content analysis and skimming. IEEE Signal Proc. Magaz., 23, 2(March 2006), 79--89.
• 14: Li, Z., Schuster, G. M., and Katsaggelos, A. K. 2005. Rate-distortion optimal video summary generation. IEEE Trans. Image Proc. 14, 10(Oct. 2005), 1550--1560.
• 15: Tao Wang , Yue Gao , Jianguo Li , Patricia P. Wang , Xiaofeng Tong , Wei Hu , Yimin Zhang , Jianmin Li, THU-ICRC at rush summarization of TRECVID 2007, Proceedings of the international workshop on TRECVID video summarization, p.79-83, September 28-28, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290031.1290046]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Abstracting methods

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Video (e.g., tape, disk, DVI); Evaluation/methodology

General Terms:
Algorithms, Experimentation, Human Factors

Keywords:
adaptive clustering, rush video summarization, spatio-temporal signature, trecvid