Proscenium: a framework for spatio-temporal video editing

ABSTRACT

We present an approach to video editing where movie sequences are treated as spatio-temporal volumes that can be sheered and warped under user control. This simple capability enables new video editing operations that support complex postproduction modifications, such as object removal and/or changes in camera motion. Our methods do not rely on complicated and error-prone image analysis or computer vision methods. Moreover, they facilitate an editing approach to video that is similar to standard image-editing tasks. Central to our system is a movie representation framework called Proscenium that supports efficient queries and operations on spatio-temporal volumes while maintaining the original source content. We have adopted a graph-based lazy-evaluation model in order to support interactive visualizations, complex data modifications, and efficient processing of large spatio-temporal volumes.

REFERENCES

• 1: Adobe After Effects 5.5. Adobe Systems Inc.http://www.adobe.com.
• 2: Adobe Photoshop 7.0. Adobe Systems Inc. http://www.adobe.com.
• 3: Adobe Premiere 6.5. Adobe Systems Inc. http://www.adobe.com.
• 4: Alias|Wavefront Maya 4.5. Alias|Wavefront. http://www.alias.com.
• 5: Apple Final Cut Pro 3. Apple Computer, Inc, http://www.apple.com.
• 6: Apple QuickTime 6. Apple Computer, Inc, http://www.apple.com.
• 7: Avid Media Composer. Avid Technology, Inc, http://www.avid.com.
• 8: AVS/Express 6.0, Advanced Visual Systems, Inc, http://www.avs.com.
• 9: Bolles, R.C., Baker, H.H., and Marimont, D.H. Epipolar-Plane Image Analysis: An Approach to Determining Structure from Motion. International Journal of Computer Vision, 1, 1, pp. 7--55, 1987.
• 10: Buehler, C., Bosse, M., and McMillan, L., Non-Metric Image-Based Rendering for Video Stabilization. In Proceedings of CVPR 2001, pp. 609--614, 2001.
• 11: Yung-Yu Chuang , Aseem Agarwala , Brian Curless , David H. Salesin , Richard Szeliski, Video matting of complex scenes, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 12: Sidney Fels , Eric Lee , Kenji Mase, Techniques for interactive video cubism (poster session), Proceedings of the eighth ACM international conference on Multimedia, p.368-370, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.354535]
• 13: Erich Gamma , Richard Helm , Ralph Johnson , John Vlissides, Design patterns: elements of reusable object-oriented software, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1995
• 14: Klein, A., Sloan, P.P., Finkelstein, A., and Cohen, M.F. Video Cubism, Microsoft Research Technical Report MSR-TR-2001-45, 2001.
• 15: Allison W. Klein , Peter-Pike J. Sloan , Adam Finkelstein , Michael F. Cohen, Stylized video cubes, Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 21-22, 2002, San Antonio, Texas [doi>10.1145/545261.545264]
• 16: Mayer-Patel, K. and Rowe, L.A. Design and Performance of the Berkeley Continuous Media Toolkit, Multimedia Computing and Networking 1997, Proc. SPIE 3020, pp. 194--206, 1997.
• 17: Microsoft DirectShow (DirectX 9.0). Microsoft Corporation, http://www.microsoft.com/directx.
• 18: Thomas Porter , Tom Duff, Compositing digital images, Proceedings of the 11th annual conference on Computer graphics and interactive techniques, p.253-259, January 1984
• 19: William K. Pratt , Jeff Dunn , Daryl Huff, Developing visual applications: XIL: an imaging foundation library, Prentice-Hall, Inc., Upper Saddle River, NJ, 1997
• 20: William H. Press , Saul A. Teukolsky , William T. Vetterling , Brian P. Flannery, Numerical recipes in C (2nd ed.): the art of scientific computing, Cambridge University Press, New York, NY, 1992
• 21: Wang, J.Y.A. and Adelson, E.A. Layered Representation for Motion Analysis. In Proceedings of CVPR93, pp. 361--366, 1993.
• 22: George Wolberg, Digital Image Warping, IEEE Computer Society Press, Los Alamitos, CA, 1994
• 23: Matthias Zwicker , Mark Pauly , Oliver Knoll , Markus Gross, Pointshop 3D: an interactive system for point-based surface editing, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Video (e.g., tape, disk, DVI)

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.4 Graphics Utilities
          Subjects: Graphics editors
      I.3.6 Methodology and Techniques
          Subjects: Graphics data structures and data types

General Terms:
Design, Management, Performance

Keywords:
feature removal, feature selection, multimedia framework, special effects, video editing, video layers, video stabilization