Finding paths through the world's photos

ABSTRACT

When a scene is photographed many times by different people, the viewpoints often cluster along certain paths. These paths are largely specific to the scene being photographed, and follow interesting regions and viewpoints. We seek to discover a range of such paths and turn them into controls for image-based rendering. Our approach takes as input a large set of community or personal photos, reconstructs camera viewpoints, and automatically computes orbits, panoramas, canonical views, and optimal paths between views. The scene can then be interactively browsed in 3D using these controls or with six degree-of-freedom free-viewpoint control. As the user browses the scene, nearby views are continuously selected and transformed, using control-adaptive reprojection techniques.

REFERENCES

• 1: Daniel G. Aliaga , Ingrid Carlbom, Plenoptic stitching: a scalable method for reconstructing 3D interactive walk throughs, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.443-450, August 2001 [doi>10.1145/383259.383311]
• 2: F. L. Bookstein, Principal Warps: Thin-Plate Splines and the Decomposition of Deformations, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.11 n.6, p.567-585, June 1989 [doi>10.1109/34.24792]
• 3: Chris Buehler , Michael Bosse , Leonard McMillan , Steven Gortler , Michael Cohen, Unstructured lumigraph rendering, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.425-432, August 2001 [doi>10.1145/383259.383309]
• 4: Shenchang Eric Chen , Lance Williams, View interpolation for image synthesis, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.279-288, August 02-06, 0093, Anaheim, CA [doi>10.1145/166117.166153]
• 5: Shenchang Eric Chen, QuickTime VR: an image-based approach to virtual environment navigation, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.29-38, September 1995 [doi>10.1145/218380.218395]
• 6: Paul E. Debevec , Camillo J. Taylor , Jitendra Malik, Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.11-20, August 1996 [doi>10.1145/237170.237191]
• 7: Drucker, S. M., and Zeltzer, D. 1994. Intelligent camera control in a virtual environment. In Proc. of Graphics Interface, 190--199.
• 8: Boris Epshtein , Eyal Ofek , Yonatan Wexler , Pusheng Zhang, Hierarchical photo organization using geo-relevance, Proceedings of the 15th annual ACM international symposium on Advances in geographic information systems, November 07-09, 2007, Seattle, Washington [doi>10.1145/1341012.1341036]
• 9: Martin A. Fischler , Robert C. Bolles, Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography, Readings in computer vision: issues, problems, principles, and paradigms, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1987
• 10: Tinsley A. Galyean, Guided navigation of virtual environments, Proceedings of the 1995 symposium on Interactive 3D graphics, p.103-ff., April 09-12, 1995, Monterey, California, United States [doi>10.1145/199404.199421]
• 11: Goesele, M., Snavely, N., Seitz, S. M., Curless, B., and Hoppe, H. 2007. Multi-view stereo for community photo collections. In Proc. Int. Conf. on Computer Vision.
• 12: Steven J. Gortler , Radek Grzeszczuk , Richard Szeliski , Michael F. Cohen, The lumigraph, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.43-54, August 1996 [doi>10.1145/237170.237200]
• 13: Kanade, T., 2001. Carnegie Mellon goes to the Superbowl. http://www.ri.cmu.edu/events/sb35/tksuperbowl.html.
• 14: Kang, S. B., Sloan, P.-P., and Seitz, S. M. 2000. Visual tunnel analysis for visibility prediction and camera planning. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 2195--2202.
• 15: Marc Levoy , Pat Hanrahan, Light field rendering, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.31-42, August 1996 [doi>10.1145/237170.237199]
• 16: Andrew Lippman, Movie-maps: An application of the optical videodisc to computer graphics, Proceedings of the 7th annual conference on Computer graphics and interactive techniques, p.32-42, July 14-18, 1980, Seattle, Washington, United States
• 17: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 18: Leonard McMillan , Gary Bishop, Plenoptic modeling: an image-based rendering system, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.39-46, September 1995 [doi>10.1145/218380.218398]
• 19: Marc Pollefeys , Luc Van Gool , Maarten Vergauwen , Frank Verbiest , Kurt Cornelis , Jan Tops , Reinhard Koch, Visual Modeling with a Hand-Held Camera, International Journal of Computer Vision, v.59 n.3, p.207-232, September-October 2004 [doi>10.1023/B:VISI.0000025798.50602.3a]
• 20: Steven M. Seitz , Charles R. Dyer, View morphing, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.21-30, August 1996 [doi>10.1145/237170.237196]
• 21: Heung-Yeung Shum , Li-Wei He, Rendering with concentric mosaics, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.299-306, July 1999 [doi>10.1145/311535.311573]
• 22: Simon, I., Snavely, N., and Seitz, S. M. 2007. Scene summarization for online image collections. In Proc. Int. Conf. on Computer Vision.
• 23: Noah Snavely , Steven M. Seitz , Richard Szeliski, Photo tourism: exploring photo collections in 3D, ACM SIGGRAPH 2006 Papers, July 30-August 03, 2006, Boston, Massachusetts
• 24: Snavely, N., Seitz, S. M., and Szeliski, R. 2008. Skeletal sets for efficient structure from motion. In Proc. Computer Vision and Pattern Recognition (to appear).
• 25: Sutherland, I. E. 1968. A head-mounted three dimensional display. In Proc. Fall Joint Computer Conf., 757--764.
• 26: Desney S. Tan , George G. Robertson , Mary Czerwinski, Exploring 3D navigation: combining speed-coupled flying with orbiting, Proceedings of the SIGCHI conference on Human factors in computing systems, p.418-425, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365307]
• 27: C. J. Taylor, VideoPlus: A Method for Capturing the Structure and Appearance of Immersive Environments, IEEE Transactions on Visualization and Computer Graphics, v.8 n.2, p.171-182, April 2002 [doi>10.1109/2945.998669]
• 28: Matthew Uyttendaele , Antonio Criminisi , Sing Bing Kang , Simon Winder , Richard Szeliski , Richard Hartley, Image-Based Interactive Exploration of Real-World Environments, IEEE Computer Graphics and Applications, v.24 n.3, p.52-63, May 2004 [doi>10.1109/MCG.2004.1297011]
• 29: Colin Ware , Steven Osborne, Exploration and virtual camera control in virtual three dimensional environments, Proceedings of the 1990 symposium on Interactive 3D graphics, p.175-183, February 1990, Snowbird, Utah, United States