An interactive comic book presentation for exploring video

ABSTRACT

This paper presents a method for generating compact pictorial summarizations of video. We developed a novel approach for selecting still images from a video suitable for summarizing the video and for providing entry points into it. Images are laid out in a compact, visually pleasing display reminiscent of a comic book or Japanese manga. Users can explore the video by interacting with the presented summary. Links from each keyframe start video playback and/or present additional detail. Captions can be added to presentation frames to include commentary or descriptions such as the minutes of a recorded meeting. We conducted a study to compare variants of our summarization technique. The study participants judged the manga summary to be significantly better than the other two conditions with respect to their suitability for summaries and navigation, and their visual appeal.

REFERENCES

• 1: Cherfaoui, M. and Bertin, C. (1994). Two-Stage Strategy for Indexing and Presenting Video. In Proc. Storage and Retrieval for Still Image and Video Databases II, SPIE 2185, San Jose, CA, pp. 174-184.
• 2: Michael G. Christel , Michael A. Smith , C. Roy Taylor , David B. Winkler, Evolving video skims into useful multimedia abstractions, Proceedings of the SIGCHI conference on Human factors in computing systems, p.171-178, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274670]
• 3: Wei Ding , Gary Marchionini , Dagobert Soergel, Multimodal surrogates for video browsing, Proceedings of the fourth ACM conference on Digital libraries, p.85-93, August 11-14, 1999, Berkeley, California, United States [doi>10.1145/313238.313266]
• 4: Ferman, A.M. and Tekalp, A.M. (1997). Multiscale Content Extraction and Representation for Video Indexing. In Proc. Multimedia Storage and Archiving Systems II, SPIE 3229, (Dallas, TX), pp. 23-31.
• 5: Borko Furht , Stephen W. Smoliar , Hongjiang Zhang, Video and image processing in multimedia systems, Kluwer Academic Publishers, Norwell, MA, 1995
• 6: Girgensohn, A., Boreczky, J., Wilcox, L., and Foote, J. (1999). Facilitating Video Access by Visualizing Automatic Analysis. In Proc. INTERACT'99, pp. 205-213.
• 7: Huang, Q., Liu, Z. and Rosenberg, A. (1999). Automated Semantic Structure Reconstruction and Representation Generation for Broadcast News. in Proc. IS&T/SPIE Conference on Storage and Retrieval for Image and Video Databases VII, Vol. 3656, pp. 50-62.
• 8: Edie Rasmussen, Clustering algorithms, Information retrieval: data structures and algorithms, Prentice-Hall, Inc., Upper Saddle River, NJ, 1992
• 9: Behzad Shahraray , David C. Gibbon, Automated authoring of hypermedia documents of video programs, Proceedings of the third ACM international conference on Multimedia, p.401-409, November 05-09, 1995, San Francisco, California, United States [doi>10.1145/217279.215304]
• 10: Snedecor, G.W. and Cochran, W.G. (1989). Statistical Methods. Ames: Iowa State University Press.
• 11: Yukinobu Taniguchi , Akihito Akutsu , Yoshinobu Tonomura, PanoramaExcerpts: extracting and packing panoramas for video browsing, Proceedings of the fifth ACM international conference on Multimedia, p.427-436, November 09-13, 1997, Seattle, Washington, United States [doi>10.1145/266180.266396]
• 12: Shingo Uchihashi , Jonathan Foote , Andreas Girgensohn , John Boreczky, Video Manga: generating semantically meaningful video summaries, Proceedings of the seventh ACM international conference on Multimedia (Part 1), p.383-392, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319463.319654]
• 13: Yeo, B.-L. and Yeung, M. (1998). Classification, Simplification and Dynamic Visualization of Scene Transition Graphs for Video Browsing. In Proc. IS&T/SPIE Electronic Imaging '98: Storage and Retrieval for Image and Video Databases VI, pp. 60-70.
• 14: Yeung, M. and Yeo, B.-L. (1997). Video Visualization for Compact Presentation and Fast Browsing of Pictorial Content. IEEE Trans. Circuits and Sys. for Video Technology, 7(5), pp. 771-785.
• 15: H. J. Zhang , C. Y. Low , S. W. Smoliar , J. H. Wu, Video parsing, retrieval and browsing: an integrated and content-based solution, Proceedings of the third ACM international conference on Multimedia, p.15-24, November 05-09, 1995, San Francisco, California, United States [doi>10.1145/217279.215068]
• 16: Zhuang, Y., Rui, Y., Huang, T.S., and Mehrotra, S. (1998). Adaptive Key Frame Extraction Using Unsupervised Clustering. In Proc. ICIP '98, Vol. I, pp. 866- 870.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Video (e.g., tape, disk, DVI)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION

General Terms:
Design, Experimentation, Human Factors, Management, Measurement, Performance, Theory

Keywords:
keyframe extraction, video browsing, video summarization