Creating map-based storyboards for browsing tour videos

ABSTRACT

Watching a long unedited video is usually a boring experience. In this paper we examine a particular subset of videos, tour videos, in which the video is captured by walking about with a running camera with the goal of conveying the essence of some place. We present a system that makes the process of sharing and watching a long tour video easier, less boring, and more informative. To achieve this, we augment the tour video with a map-based storyboard, where the tour path is reconstructed, and coherent shots at different locations are directly visualized on the map. This allows the viewer to navigate the video in the joint location-time space. To create such a storyboard we employ an automatic pre-processing component to parse the video into coherent shots, and an authoring tool to enable the user to tie the shots with landmarks on the map. The browser-based viewing tool allows users to navigate the video in a variety of creative modes with a rich set of controls, giving each viewer a unique, personal viewing experience. Informal evaluation shows that our approach works well for tour videos compared with conventional media players.

REFERENCES

• 1: M. Cooper and J. Foote. Scene boundary detection via video self-similarity analysis. Image Processing, 2001. Proceedings. 2001 International Conference on, 3:378--381 vol.3, 2001.
• 2: Jonathan Foote , John Boreczhy , Andreas Girgensohn , Lynn Wilcox, An intelligent media browser using automatic multimodal analysis, Proceedings of the sixth ACM international conference on Multimedia, p.375-380, September 13-16, 1998, Bristol, United Kingdom [doi>10.1145/290747.290804]
• 3: Andreas Girgensohn , John Boreczky , Patrick Chiu , John Doherty , Jonathan Foote , Gene Golovchinsky , Shingo Uchihashi , Lynn Wilcox, A semi-automatic approach to home video editing, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.81-89, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354415]
• 4: Daniel R. Goldman , David H. Salesin , Brian Curless, A framework for video annotation, visualization, and interaction, University of Washington, Seattle, WA, 2007
• 5: Rieko Kadobayashi , Katsumi Tanaka, 3D viewpoint-based photo search and information browsing, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076158]
• 6: Anita Komlodi , Gary Marchionini, Key frame preview techniques for video browsing, Proceedings of the third ACM conference on Digital libraries, p.118-125, June 23-26, 1998, Pittsburgh, Pennsylvania, United States [doi>10.1145/276675.276688]
• 7: Y. Li, T. Zhang, and D. Tretter. An overview of video abstraction techniques. Technical Report HPL-2001-191, HP Laboratory, July 2001.
• 8: Rainer Lienhart, Abstracting home video automatically, Proceedings of the seventh ACM international conference on Multimedia (Part 2), p.37-40, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319878.319888]
• 9: Andrew Lippman, Movie-maps: An application of the optical videodisc to computer graphics, ACM SIGGRAPH Computer Graphics, v.14 n.3, p.32-42, July 1980
• 10: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 11: J. Matas, O. Chum, M. Urban, and T. Pajdla. Robust wide baseline stereo from maximally stable extremal regions. In Proc. of the British Machine Vision Conference, pages 384--393, 2002.
• 12: Krystian Mikolajczyk , Cordelia Schmid, Scale & Affine Invariant Interest Point Detectors, International Journal of Computer Vision, v.60 n.1, p.63-86, October 2004 [doi>10.1023/B:VISI.0000027790.02288.f2]
• 13: Mor Naaman , Yee Jiun Song , Andreas Paepcke , Hector Garcia-Molina, Automatic organization for digital photographs with geographic coordinates, Proceedings of the 4th ACM/IEEE-CS joint conference on Digital libraries, June 07-11, 2004, Tuscon, AZ, USA [doi>10.1145/996350.996366]
• 14: J. Nam and A. H. Tewfik. Video abstract of video. In Proc. of IEEE 3rdWorkshop on Multimedia Signal Processing, pages 117--122, 1999.
• 15: Silvia Pfeiffer , Rainer Lienhart , Stephan Fischer , Wolfgang Effelsberg, Abstracting Digital Movies Automatically, University of Mannheim, 1996
• 16: Michael A. Smith Takeo Kanade, Video Skimming and Characterization through the Combination of Image and Language Understanding, Proceedings of the 1998 International Workshop on Content-Based Access of Image and Video Databases (CAIVD '98), p.61, January 03-03, 1998
• 17: Noah Snavely , Steven M. Seitz , Richard Szeliski, Photo tourism: exploring photo collections in 3D, ACM SIGGRAPH 2006 Papers, July 30-August 03, 2006, Boston, Massachusetts
• 18: Yoshinobu Tonomura , Akihito Akutsu , Kiyotaka Otsuji , Toru Sadakata, VideoMAP and VideoSpaceIcon: tools for anatomizing video content, Proceedings of the INTERCHI '93 conference on Human factors in computing systems, p.131-136, May 1993, Amsterdam, The Netherlands
• 19: Kentaro Toyama , Ron Logan , Asta Roseway, Geographic location tags on digital images, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957046]
• 20: Ba Tu Truong , Svetha Venkatesh, Video abstraction: A systematic review and classification, ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), v.3 n.1, p.3-es, February 2007 [doi>10.1145/1198302.1198305]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

General Terms:
Design, Human Factors

Keywords:
map, storyboard, video browsing, video summarization