Dynamic key frame presentation techniques for augmenting video browsing

ABSTRACT

Because of unique temporal and spatial properties of video data, different techniques for summarizing videos have been proposed. Key frames extracted directly from video inform users about content without requiring them to view the entire video. As part of ongoing work to develop video browsing interfaces, several interface displays based on key frames were investigated. Variations on dynamic key frame "slide shows" were examined and compared to a static key frame "filmstrip" display. The slide show mechanism displays key frames in rapid succession and is designed to facilitate visual browsing by exploiting human perceptual capabilities. User studies were conducted in a series of three experiments. Key frame display rate, number of simultaneous displays, and user perception were investigated as a function of user performance in object recognition and gist determination tasks. No significant performance degradation was detected at display rates up to 8 key frames per second, but performance degraded significantly at higher rates. Performance on gist determination tasks degraded less severely than performance on object recognition tasks as display rates increased. Furthermore, gist determination performance dropped significantly between three and four simultaneous slide shows in a single display. Users generally preferred key frame filmstrips to dynamic displays, although objective measures of performance were mixed. Implications for visual interface design and further questions for future research are provided.

REFERENCES

• 1: S. S. Beauchemin , J. L. Barron, The computation of optical flow, ACM Computing Surveys (CSUR), v.27 n.3, p.433-466, Sept. 1995 [doi>10.1145/212094.212141]
• 2: Boyce, S. J., A. Pollatsek, and K. Rayner. (1989). Effect of background information on object identification. Journal of Experimental Psychology: Human Perception and Performance, 15(3): 556--566.
• 3: Michael G. Christel , David B. Winkler , C. Roy Taylor, Improving Access to a Digital Video Library, Proceedings of the IFIP TC13 Interantional Conference on Human-Computer Interaction, p.524-531, July 14-18, 1997
• 4: Ding, W., G. Marchionini, and T. Tse. (1997). Previewing video data: Browsing key frames at high rates using a video slide show interface. Proceedings of the International Symposium on Research, Development, and Practice in Digital Libraries (ISDL '97), Tsukuba, Japan.
• 5: Ellis, H. C. and R. R. Hunt. (1989). Fundamentals of human memory and cognition. Wm. C. Brown Publishers: Dubuque, IA.
• 6: Elliot, E. (1993). Watch, grab, arrange, see: Thinking with motion images via streams and collages. MSVS Thesis Document. Cambridge, MA: MIT Media Lab.
• 7: Christopher G. Healey , Kellogg S. Booth , James T. Enns, High-speed visual estimation using preattentive processing, ACM Transactions on Computer-Human Interaction (TOCHI), v.3 n.2, p.107-135, June 1996 [doi>10.1145/230562.230563]
• 8: Komlodi, A. (1997). Visual surrogates for motion picture documents: Presentation techniques for key frames. CLIS-TR-97-15, College Park, MD: Digital Library Research Group (http://www.glue.umd.edu/~dlrg/).
• 9: Gary Marchionini, Information seeking in electronic environments, Cambridge University Press, New York, NY, 1995
• 10: O'Connor, B. C. (1991). Selecting key frames of moving image documents: A digital environment for analysis and navigation. Microcomputers for Information Management, 8(2), 119--133.
• 11: Potter, M. C. (1976). Short-term conceptual memory for pictures. Journal of Experimental Psychology: Human Learning and Memory, 2(5), 509--522.
• 12: Laura Slaughter , Ben Shneiderman , Gary Marchionini, Comprehension and Object Recognition Capabilities for Presentations of Simultaneous Video Key Frame Surrogates, Proceedings of the First European Conference on Research and Advanced Technology for Digital Libraries, p.41-54, September 01-03, 1997
• 13: Laura Teodosio , Walter Bender, Salient video stills: content and context preserved, Proceedings of the first ACM international conference on Multimedia, p.39-46, August 02-06, 1993, Anaheim, California, United States [doi>10.1145/166266.166270]
• 14: Howard D. Wactlar , Takeo Kanade , Michael A. Smith , Scott M. Stevens, Intelligent Access to Digital Video: Informedia Project, Computer, v.29 n.5, p.46-52, May 1996 [doi>10.1109/2.493456]
• 15: Wickens, C. D. (1992). Engineering psychology and human performance. Second Edition. New York: HarperCollins.
• 16: Yeung, M. M, B. L. Yeo, W. Wolf, and B. Liu. (1995). Video browsing using clustering and scene transition on compressed sequences. Proceedings of Multimedia Computing and Networking, San Jose.
• 17: Yow, D, B. L. Yeo, M. M. Yeung, and B. Liu. (1995). Analysis and presentation of soccer highlights from digital video. Proceedings of the Second Asian Conference on Computer Vision (ACCV '95).
• 18: Hongjiang Zhang , Chien Yong Low , Stephen W. Smoliar, Video parsing and browsing using compressed data, Multimedia Tools and Applications, v.1 n.1, p.89-111, March 1995 [doi>10.1007/BF01261227]

INDEX TERMS

Keywords:
display rate, divided attention, dynamic displays, interface design, key frames, representations, video browsing