Dynamic storyboards for video content summarization

ABSTRACT

We propose an innovative, general purpose, approach to the selection and hierarchical representation of key frames of a video sequence for video summarization. In the first stage the shot detection module performs the video structural analysis; in the second stage the key frame extraction module creates the visual summary; and the last stage the summary post-processing module, after a pre-a classification aimed remove meaningless key frames, create a multilevel storyboard that the user may browse.

REFERENCES

• 1: Agraim P., Zhang H., Petkovic D., "Content-Based Representation and Retrieval of Visual Media: A State of the Art review", Multimedia Tools and Applications, vol. 3, pp. 179--202, 1996.
• 2: Breiman L., Friedman J. H., Olshen R. A.and Stone C. J., Classification and Regression Trees, (Wadsworth and Brooks/Cole), 1984.
• 3: Chetverikov D. and Szabo Zs. "A Simple and Efficient Algorithm for Detection of High Curvature Points in Planar Curves", Proc. 23rd Workshop of the Austrian Pattern Recognition Group, pp. 175--184, 1999.
• 4: Ciocca G., Gagliardi I., Schettini R. "Quicklook2: An Integrated Multimedia System", International Journal of Visual Languages and Computing, Special issue on Querying Multiple Data Sources, vol. 12. pp. 81--103, 2001.
• 5: Corchs S., Deco G., "Large-scale neural model for visual attention: integration of experimental single-cell and fMRI data", Cerebral Cortex, vol. 12, pp. 339--348, 2002.
• 6: Deng Y.N., Henny C., Moore M.S., Manjunath B.S., "Peer group filtering and perceptual color quantization", IEEE Int. Symposium on Circuits and Systems, vol. 4, pp. 21--24, 1999.
• 7: Divakaran A., Cabasson R., "Content Based Browsing System for Personal Video Recorders", Proc. Int. Conference on Consumer Electronics, pp. 114--115, 2002.
• 8: Fernando W. A. C., Canaharajah C. N., Bull D. R., "Fade-In and Fade-Out Detection in Video Sequences Using Histograms", Proc. ISCAS 2000 -- IEEE International Symposium on Circuits and System, IV 709--712, Geneva, Switzerland, 2000.
• 9: Fisher R.A., "The use of multiple measures in taxonomic problems", Ann. Eugenics, vol. 7, pp.179--188, 1936.
• 10: Rafael C. Gonzalez , Richard E. Woods, Digital Image Processing, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1992
• 11: Idris F., Panchanathan S., "Storage and retrieval of compressed images using wavelet vector quantization", Journal of Visual Languages and Computing,vol. 8, pp. 289--301, 1997.
• 12: Kenney C., Deng Y.N., Manjunath B.S., Hewer G., "Peer Group image enhancement", IEEE Trans. Image Processing, 10(2), 326--334, 2001.
• 13: Lee J.H., Lee G.G., Kim W.Y. "Automatic Video Summarization Tool using MPEG-7 Descriptors for Personal Video Recorder", IEEE Transaction on Consumer Electronics, Vol. 49, No. 3, pp.742--249, 2003
• 14: Hong Lu , Yap-Peng Tan, Unsupervised clustering of dominant scenes in sports video, Pattern Recognition Letters, v.24 n.15, p.2651-2662, November 2003 [doi>10.1016/S0167-8655(03)00108-9]
• 15: D. E. Rumelhart , G. E. Hinton , R. J. Williams, Learning internal representations by error propagation, Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations, MIT Press, Cambridge, MA, 1986
• 16: Schettini R., Brambilla C., Cusano C., Ciocca G., "Automatic classification of digital photographs based on decision forests", Int. J. of Pattern Recognition and Artificial Intelligence, vol. 18(5), pp. 819--846, 2004.
• 17: Scheunders P., Livens S., Van de Wouwer G., Vautrot P., Van Dyck D. "Wavelet-based texture analysis", Int. Journal Computer Science and Information management, vol. 1(2), pp. 22--34, 1998.
• 18: URL: Archivio di Etnografie e Storia Sociale, Regione Lombardia (AESS Archive): http://aess.itc.cnr.it/index.htm.
• 19: Vercauteren D., Quality assurance for digital camera images, EPFL-IVRG Project Report, 2003.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Abstracting methods

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
        I.4.10 Image Representation
            Subjects: Multidimensional
  I.5 PATTERN RECOGNITION
      I.5.3 Clustering

General Terms:
Algorithms

Keywords:
dynamic video summaries, multilevel visual summaries, summary post-processing, supervised and unsupervised classification