Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

This paper describes a mostly automatic method for taking the output of a single panning video camera and creating a panoramic video texture (PVT): a video that has been stitched into a single, wide field of view and that appears to play continuously and indefinitely. The key problem in creating a PVT is that although only a portion of the scene has been imaged at any given time, the output must simultaneously portray motion throughout the scene. Like previous work in video textures, our method employs min-cut optimization to select fragments of video that can be stitched together both spatially and temporally. However, it differs from earlier work in that the optimization must take place over a much larger set of data. Thus, to create PVTs, we introduce a dynamic programming step, followed by a novel hierarchical min-cut optimization algorithm. We also use gradient-domain compositing to further smooth boundaries between video fragments. We demonstrate our results with an interactive viewer in which users can interactively pan and zoom on high-resolution PVTs.

REFERENCES

• 1: Aseem Agarwala , Mira Dontcheva , Maneesh Agrawala , Steven Drucker , Alex Colburn , Brian Curless , David Salesin , Michael Cohen, Interactive digital photomontage, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004
• 2: James R. Bergen , P. Anandan , Keith J. Hanna , Rajesh Hingorani, Hierarchical Model-Based Motion Estimation, Proceedings of the Second European Conference on Computer Vision, p.237-252, May 19-22, 1992
• 3: M. Brown , D. G. Lowe, Recognising Panoramas, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.1218, October 13-16, 2003
• 4: Shenchang Eric Chen, QuickTime VR: an image-based approach to virtual environment navigation, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.29-38, September 1995 [doi>10.1145/218380.218395]
• 5: Adam Finkelstein , Charles E. Jacobs , David H. Salesin, Multiresolution video, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.281-290, August 1996 [doi>10.1145/237170.237266]
• 6: Fitzgibbon, A. W. 2001. Stochastic rigidity: Image registration for nowhere-static scenes. In Proceedings of ICCV 2001, 662--670.
• 7: William T. Freeman , Egon C. Pasztor , Owen T. Carmichael, Learning Low-Level Vision, International Journal of Computer Vision, v.40 n.1, p.25-47, Oct. 2000 [doi>10.1023/A:1026501619075]
• 8: Irani, M., and Anandan, P. 1998. Video indexing based on mosaic representation. Proceedings of IEEE 86, 5, 905--921.
• 9: Don Kimber , Jonathan Foote , Surapong Lertsithichai, FlyAbout: spatially indexed panoramic video, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500192]
• 10: Vladimir Kolmogorov , Ramin Zabih, What Energy Functions Can Be Minimized via Graph Cuts?, Proceedings of the 7th European Conference on Computer Vision-Part III, p.65-81, May 28-31, 2002
• 11: Vivek Kwatra , Arno Schödl , Irfan Essa , Greg Turk , Aaron Bobick, Graphcut textures: image and video synthesis using graph cuts, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 12: Shree K. Nayar, Catadioptric Omnidirectional Camera, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.482, June 17-19, 1997
• 13: Ulrich Neumann , Thomas Pintaric , Albert Rizzo, Immersive panoramic video, Proceedings of the eighth ACM international conference on Multimedia, p.493-494, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.376408]
• 14: Patrick Pérez , Michel Gangnet , Andrew Blake, Poisson image editing, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 15: Point Grey Research, 2005. http://ptgrey.com.
• 16: Alex Rav-Acha , Yael Pritch , Dani Lischinski , Shmuel Peleg, Dynamosaics: Video Mosaics with Non-Chronological Time, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1, p.58-65, June 20-26, 2005 [doi>10.1109/CVPR.2005.137]
• 17: Arno Schödl , Richard Szeliski , David H. Salesin , Irfan Essa, Video textures, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.489-498, July 2000 [doi>10.1145/344779.345012]
• 18: Richard Szeliski , Heung-Yeung Shum, Creating full view panoramic image mosaics and environment maps, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.251-258, August 1997 [doi>10.1145/258734.258861]
• 19: Bill Triggs , Philip F. McLauchlan , Richard I. Hartley , Andrew W. Fitzgibbon, Bundle Adjustment - A Modern Synthesis, Proceedings of the International Workshop on Vision Algorithms: Theory and Practice, p.298-372, September 21-22, 1999
• 20: Uyttendaele, M., Criminisi, A., Kang, S. B., Winder, S. A. J., Hartley, R., and Szeliski, R. 2004. High-quality image-based interactive exploration of real-world environments. IEEE Computer Graphics and Applications 24, 3, 52--63.
• 21: Hongcheng Wang , Ramesh Raskar , Narendra Ahuja, Seamless Video Editing, Proceedings of the Pattern Recognition, 17th International Conference on (ICPR'04) Volume 3, p.858-861, August 23-26, 2004 [doi>10.1109/ICPR.2004.806]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Video analysis

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Texture
      I.2.8 Problem Solving, Control Methods, and Search
          Subjects: Dynamic programming

General Terms:
Algorithms, Performance

Keywords:
panoramas, video textures, video-based rendering