Rushes summarization with self-organizing maps

ABSTRACT

In this paper, we describe our approach for video summarization that was applied to the BBC rushes material as part of the TRECVID 2007 evaluations. The method consists of initial shot boundary detection followed by shot similarity assessment and pruning, with both stages implemented using multiple parallel Self-Organizing Maps and within our content-based multimedia information retrieval and analysis framework named PicSOM. The results indicate that our approach can be successfully applied to rushes summarization. Compared to other submissions, our method resulted in the overall shortest summaries with close to median performance in the fraction of ground-truth inclusions found.

REFERENCES

• 1: D. Chai and K. N. Ngan. Face segmentation using skin-color map in videophone applications. IEEE Transactions on Circuits and Systems for Video Technology, 9(4):551--564, June 1999.
• 2: ISO/IEC JTC 1. Multimedia content description interface - Part 3: Visual, 2002. 15938--3:2002(E).
• 3: Teuvo Kohonen, The 'Neural' Phonetic Typewriter, Computer, v.21 n.3, p.11-22, March 1988 [doi>10.1109/2.28]
• 4: T. Kohonen , M. R. Schroeder , T. S. Huang, Self-Organizing Maps, Springer-Verlag New York, Inc., Secaucus, NJ, 2001
• 5: J. Laaksonen, M. Koskela, and E. Oja. PicSOM--Self-organizing image retrieval with MPEG-7 content descriptions. IEEE Transactions on Neural Networks, 13(4):841--853, July 2002.
• 6: Jorma T. Laaksonen , J. Markus Koskela , Erkki Oja, Class distribution on SOM surfaces for feature extraction and object retrieval, Neural Networks, v.17 n.8-9, p.1121-1133, October/November 2004 [doi>10.1016/j.neunet.2004.07.007]
• 7: H. Muurinen and J. Laaksonen. Video segmentation and shot boundary detection using self-organizing maps. In Proceedings of 15th Scandinavian Conference on Image Analysis (SCIA 2007), pages 770--779, Aalborg, Denmark, June 2007.
• 8: Paul Over , Alan F. Smeaton , Philip Kelly, The trecvid 2007 BBC rushes summarization evaluation pilot, Proceedings of the international workshop on TRECVID video summarization, p.1-15, September 28-28, 2007, Augsburg, Bavaria, Germany [doi>10.1145/1290031.1290032]
• 9: M. Sjöberg, H. Muurinen, J. Laaksonen, and M. Koskela. PicSOM experiments in TRECVID 2006. In Proceedings of the TRECVID 2006 Workshop, Gaithersburg, MD, USA, November 2006.
• 10: Alan F. Smeaton , Paul Over , Wessel Kraaij, Evaluation campaigns and TRECVid, Proceedings of the 8th ACM international workshop on Multimedia information retrieval, October 26-27, 2006, Santa Barbara, California, USA [doi>10.1145/1178677.1178722]
• 11: V. Tryba and K. Goser. Self-organizing feature maps for process control in chemistry. In Proceedings of International Conference on Artificial Neural Networks (ICANN-91), volume 1, pages 847--852, Espoo,

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Evaluation/methodology

General Terms:
Algorithms, Experimentation

Keywords:
self-organizing map, video summarization