Photographing long scenes with multi-viewpoint panoramas

ABSTRACT

We present a system for producing multi-viewpoint panoramas of long, roughly planar scenes, such as the facades of buildings along a city street, from a relatively sparse set of photographs captured with a handheld still camera that is moved along the scene. Our work is a significant departure from previous methods for creating multi-viewpoint panoramas, which composite thin vertical strips from a video sequence captured by a translating video camera, in that the resulting panoramas are composed of relatively large regions of ordinary perspective. In our system, the only user input required beyond capturing the photographs themselves is to identify the dominant plane of the photographed scene; our system then computes a panorama automatically using Markov Random Field optimization. Users may exert additional control over the appearance of the result by drawing rough strokes that indicate various high-level goals. We demonstrate the results of our system on several scenes, including urban streets, a river bank, and a grocery store aisle.

REFERENCES

• 1: Aseem Agarwala , Mira Dontcheva , Maneesh Agrawala , Steven Drucker , Alex Colburn , Brian Curless , David Salesin , Michael Cohen, Interactive digital photomontage, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004
• 2: Aseem Agarwala , Ke Colin Zheng , Chris Pal , Maneesh Agrawala , Michael Cohen , Brian Curless , David Salesin , Richard Szeliski, Panoramic video textures, ACM Transactions on Graphics (TOG), v.24 n.3, July 2005
• 3: Maneesh Agrawala , Denis Zorin , Tamara Munzner, Artistic Multiprojection Rendering, Proceedings of the Eurographics Workshop on Rendering Techniques 2000, p.125-136, June 26-28, 2000
• 4: Astola, J., Haavisto, P., and Neuvo, Y. 1990. Vector median filters. Proceedings of the IEEE 78, 678--689.
• 5: Marcelo Bertalmio , Guillermo Sapiro , Vincent Caselles , Coloma Ballester, Image inpainting, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.417-424, July 2000 [doi>10.1145/344779.344972]
• 6: Yuri Boykov , Olga Veksler , Ramin Zabih, Fast Approximate Energy Minimization via Graph Cuts, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.11, p.1222-1239, November 2001 [doi>10.1109/34.969114]
• 7: M. Brown , D. G. Lowe, Unsupervised 3D Object Recognition and Reconstruction in Unordered Datasets, Proceedings of the Fifth International Conference on 3-D Digital Imaging and Modeling, p.56-63, June 13-16, 2005 [doi>10.1109/3DIM.2005.81]
• 8: J. Davis, Mosaics of Scenes with Moving Objects, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.354, June 23-25, 1998
• 9: ePaperPress, 2005. http://epaperpress.com/ptlens/.
• 10: Google, 2005. http://earth.google.com.
• 11: Rajiv Gupta , Richard I. Hartley, Linear Pushbroom Cameras, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.9, p.963-975, September 1997 [doi>10.1109/34.615446]
• 12: Richard Hartley , Andrew Zisserman, Multiple view geometry in computer visiond, Cambridge University Press, New York, NY, 2001
• 13: Huber, P. 1981. Robust statistics. John Wiley.
• 14: Kang, S. B., Szeliski, R., and Chai, J. 2001. Handling occlusions in dense multi-view stereo. In Computer Vision and Pattern Recognition (CVPR 2001), vol. 1, 103--110.
• 15: Yves Egels , Michel Kasser, Digital Photogrammetry, Taylor & Francis, Inc., Bristol, PA, 2001
• 16: Koller, M., 2004. http://www.seamlesscity.com.
• 17: Vladimir Kolmogorov , Ramin Zabih, What Energy Functions Can Be Minimized via Graph Cuts?, Proceedings of the 7th European Conference on Computer Vision-Part III, p.65-81, May 28-31, 2002
• 18: Kubovy, M. 1986. The psychology of perspective and renaissance art. Cambridge University Press.
• 19: Vivek Kwatra , Arno Schödl , Irfan Essa , Greg Turk , Aaron Bobick, Graphcut textures: image and video synthesis using graph cuts, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 20: Levin, G., 2005. An informal catalogue of slit-scan video artworks. http://www.flong.com/writings/lists/list_slit_scan.html.
• 21: Marc Levoy , Billy Chen , Vaibhav Vaish , Mark Horowitz , Ian McDowall , Mark Bolas, Synthetic aperture confocal imaging, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004
• 22: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 23: Mitsunaga, T., and Nayar, S. K. 1999. Radiometric self calibration. In Computer Vision and Pattern Recognition (CVPR '99), 374--380.
• 24: Shmuel Peleg , Benny Rousso , Alex Rav-Acha , Assaf Zomet, Mosaicing on Adaptive Manifolds, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.10, p.1144-1154, October 2000 [doi>10.1109/34.879794]
• 25: Patrick Pérez , Michel Gangnet , Andrew Blake, Poisson image editing, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 26: Paul Rademacher , Gary Bishop, Multiple-center-of-projection images, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.199-206, July 1998 [doi>10.1145/280814.280871]
• 27: Alex Rav-Acha , Yael Shor , Shmuel Peleg, Mosaicing with Parallax using Time Warping, Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'04) Volume 11, p.164, June 27-July 02, 2004
• 28: Augusto Roman , Gaurav Garg , Marc Levoy, Interactive Design of Multi-Perspective Images for Visualizing Urban Landscapes, Proceedings of the conference on Visualization '04, p.537-544, October 10-15, 2004 [doi>10.1109/VIS.2004.50]
• 29: Steven M. Seitz , Jiwon Kim, Multiperspective Imaging, IEEE Computer Graphics and Applications, v.23 n.6, p.16-19, November 2003 [doi>10.1109/MCG.2003.1242377]
• 30: Noah Snavely , Steven M. Seitz , Richard Szeliski, Photo tourism: exploring photo collections in 3D, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
• 31: Jian Sun , Lu Yuan , Jiaya Jia , Heung-Yeung Shum, Image completion with structure propagation, ACM Transactions on Graphics (TOG), v.24 n.3, July 2005
• 32: Richard Szeliski , Heung-Yeung Shum, Creating full view panoramic image mosaics and environment maps, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.251-258, August 1997 [doi>10.1145/258734.258861]
• 33: Uyttendaele, M., Eden, A., and Szeliski, R. 2001. Eliminating ghosting and exposure artifacts in image mosaics. In Computer Vision and Pattern Recognition (CVPR 01), 509--516.
• 34: Y. Wexler , D. Simakov, Space-Time Scene Manifolds, Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, p.858-863, October 17-20, 2005 [doi>10.1109/ICCV.2005.227]
• 35: Daniel N. Wood , Adam Finkelstein , John F. Hughes , Craig E. Thayer , David H. Salesin, Multiperspective panoramas for cel animation, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.243-250, August 1997 [doi>10.1145/258734.258859]
• 36: Yu, J., and McMillan, L. 2004. A framework for multiperspective rendering. In Proceedings of the 15th Eurographics workshop on Rendering Techniques, 61--68.
• 37: Yu, J., and McMillan, L. 2004. General linear cameras. In European Conference on Computer Vision (ECCV 04), 14--27.
• 38: Jiang Yu Zheng, Digital Route Panoramas, IEEE MultiMedia, v.10 n.3, p.57-67, July 2003 [doi>10.1109/MMUL.2003.1218257]
• 39: Zomet, A., Feldman, D., Peleg, S., and Weinshall, D. 2003. Mosaicing new views: The crossed-slits projection. IEEE Transactions on PAMI 25, 6, 741--754.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.1 Digitization and Image Capture

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.5 Computational Geometry and Object Modeling
          Subjects: Curve, surface, solid, and object representations; Geometric algorithms, languages, and systems

General Terms:
Algorithms, Design