A framework for virtual videography

ABSTRACT

There are a significant number of events that happen on a regular basis that would be worth preserving on video but for which it is impractical to use traditional video production methods. In this paper, we describe one possible way to inexpensively and unobtrusively capture and produce video in a classroom lecture environment. We discuss the importance of cinematic principles in the lecture video domain and describe guidelines that should be followed when capturing a lecture. We continue by surveying the tools provided by computer vision and computer graphics that allow us to determine syntactic information about images. Finally, we describe a way to combine these tools to create a framework for a Virtual Videography system, one that can automatically generate production quality video. This framework is based on the creation of region objects, a semantically related region of video, despite the fact that we can reliably only gather syntactic information.

REFERENCES

• 1: William H. Bares , James C. Lester, Intelligent multi-shot visualization interfaces for dynamic 3D worlds, Proceedings of the 4th international conference on Intelligent user interfaces, p.119-126, January 05-08, 1999, Los Angeles, California, United States [doi>10.1145/291080.291101]
• 2: M. Bianchi. Autoauditorium: a fully automatic, multi-camera system to televise auditorium presentations, 1998.
• 3: David Bordwell. On the History of Film Style. Harvard University Press, 1998.
• 4: David Bordwell and Kristin Thompson. Film Art: An Introduction. The McGraw-Hill Companies, Inc., 1997.
• 5: Steven M. Drucker , David Zeltzer, CamDroid: a system for implementing intelligent camera control, Proceedings of the 1995 symposium on Interactive 3D graphics, p.139-144, April 09-12, 1995, Monterey, California, United States [doi>10.1145/199404.199428]
• 6: Foveal Systems, LLC. Auto auditorium. web page, 1999-2000. www.autoauditorium.com.
• 7: William T. Freeman , David B. Anderson , Paul A. Beardsley , Chris N. Dodge , Michal Roth , Craig D. Weissman , William S. Yerazunis , Hiroshi Kage , Kazuo Kyuma , Yasunari Miyake , Ken-ichi Tanaka, Computer Vision for Interactive Computer Graphics, IEEE Computer Graphics and Applications, v.18 n.3, p.42-53, May 1998 [doi>10.1109/38.674971]
• 8: Michael Gleicher , James Masanz, Towards virtual videography (poster session), Proceedings of the eighth ACM international conference on Multimedia, p.375-378, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.354537]
• 9: Li-wei He , Michael F. Cohen , David H. Salesin, The virtual cinematographer: a paradigm for automatic real-time camera control and directing, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.217-224, August 1996 [doi>10.1145/237170.237259]
• 10: M. Hunke and A. Waibel. Face locating and tracking for human-computer interaction, 1994.
• 11: John D. Gould , John Conti , Todd Hovanyecz, Composing letters with a simulated listening typewriter, Proceedings of the 1982 conference on Human factors in computing systems, p.367-370, March 15-17, 1982, Gaithersburg, Maryland, United States [doi>10.1145/800049.801813]
• 12: Michael J. Jones and James M. Rehg. Statistical color models with application to skin detection. Technical Report CRL 98/11, Cambridge Research Laboratory, December 1998.
• 13: Shanon Ju, Michael Black, Scott Minneman, and Don Kimber. Summarization of video-taped presentations: Automatic analysis of motion and gesture. IEEE Transactions on Circuits and Systems for Video Technology, 1998.
• 14: J. F. Kelley, An iterative design methodology for user-friendly natural language office information applications, ACM Transactions on Information Systems (TOIS), v.2 n.1, p.26-41, Jan. 1984 [doi>10.1145/357417.357420]
• 15: Norman J. Medoff and Tom Tanquary. Portable Video ENG and EFP. Focal Press, fourth edition edition, 2002.
• 16: Sugata Mukhopadhyay , Brian Smith, Passive capture and structuring of lectures, Proceedings of the seventh ACM international conference on Multimedia (Part 1), p.477-487, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319463.319690]
• 17: University of Wisconsin Madison. eteach. http://eteach.engr.wisc.edu/newEteach/home.html, 1999.
• 18: Yong Rui , Liwei He , Anoop Gupta , Qiong Liu, Building an intelligent camera management system, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500145]
• 19: Eric Saund. Image mosaicing and a diagrammatic user interface for an office whiteboard scanner. http://www.parc.xerox.com/spl/members/saund/zombieboard-public.html.
• 20: Harpreet S. Sawhney , Serge Ayer, Compact Representations of Videos Through Dominant and Multiple Motion Estimation, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.18 n.8, p.814-830, August 1996 [doi>10.1109/34.531801]
• 21: Arno Schödl , Richard Szeliski , David H. Salesin , Irfan Essa, Video textures, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.489-498, July 2000 [doi>10.1145/344779.345012]
• 22: Tanveer Syeda-Mahmood and et al. Cuevideo: A system for cross-modal search and browse of video databases. In Proceedings of Computer Vision and Pattern Recognition, June 2000.
• 23: Virtual Ink Corp. Mimio. Computer Hardware Product, 2000.
• 24: Michael N. Wallick, Niels da Vitoria Lobo, and Mubarak Shah. Computer vision framework for analyzing computer and overhead projections from within video. International Journal of Computers and Their Applications, 8(2), June 2001.
• 25: J. Y. A. Wang and E. H. Adelson. Representing moving images with layers. The IEEE Transactions on Image Processing Special Issue: Image Sequence Compression, 3(5):625-638, September 1994.