Rushes summarization based on color, motion and face

ABSTRACT

In this paper, we present a method for the Rushes Summarization task which is one of tasks of TRECVID 2008. In the proposed method, first an input video is decomposed into shots by comparing consecutive frames. Then, these shots are grouped by the k-means method, using color, motion and faces as features. In the preliminary experiments, we compared three systems which employed the following feature combinations: "color", "color and motion" and "color, motion and faces". As a result, we found out that motion features and face features were effective.

Our results of Rushes Summarization 2008 were a little below the median regarding IN (inclusion ratio of ground truth) and JU (lack of junk shots), but were above the median regarding TE (pleasant tempo). Then, to improve IN and JU, we modified the method to detect clapper boards by introducing visual feature in addition to sound feature. The additional experiment regarding the modification after submission shows that it improved the results.

REFERENCES

• 1: G. Csurka, C. Bray, C. Dance, and L. Fan. Visual categorization with bags of keypoints. In Proc. of ECCV Workshop on Statistical Learning in Computer Vision, pages 1--22, 2004.
• 2: Alexander G. Hauptmann , Michael G. Christel , Wei-Hao Lin , Bryan Maher , Jun Yang , Robert V. Baron , Guang Xiang, Clever clustering vs. simple speed-up for summarizing rushes, Proceedings of the international workshop on TRECVID video summarization, p.20-24, September 28-28, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290031.1290034]
• 3: Duy-Dinh Le , Shin'ichi Satoh, National institute of informatics, japan at TRECVID 2007: BBC rushes summarization, Proceedings of the international workshop on TRECVID video summarization, p.70-73, September 28-28, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290031.1290044]
• 4: Yang Liu , Yan Liu , Yan Zhang, The Hong Kong Polytechnic University at TRECVID 2007 BBC rushes summarization, Proceedings of the international workshop on TRECVID video summarization, p.50-54, September 28-28, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290031.1290040]
• 5: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 6: B. Lucas and T. Kanade. An iterative image registration technique with an application to stereo vision. In Proc. of International Joint Conference on Artificial Intelligence, pages 674--679, 1981.
• 7: Paul Over , Alan F. Smeaton , George Awad, The trecvid 2008 BBC rushes summarization evaluation, Proceedings of the 2nd ACM TRECVid Video Summarization Workshop, p.1-20, October 31-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1463563.1463564]
• 8: P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. In Proc. IEEE Computer Vision and Pattern Recognition, volume 1, pages 511--518, 2001.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.m Miscellaneous

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Performance measures; Complexity measures

General Terms:
Algorithms, Experimentation, Performance

Keywords:
rushes summarization, trecvid, video editing