Surface light fields for 3D photography

ABSTRACT

A surface light field is a function that assigns a color to each ray originating on a surface. Surface light fields are well suited to constructing virtual images of shiny objects under complex lighting conditions. This paper presents a framework for construction, compression, interactive rendering, and rudimentary editing of surface light fields of real objects. Generalization of vector quantization and principal component analysis are used to construct a compressed representation of an object's surface light field from photographs and range scans. A new rendering algorithm achieves interactive rendering of images from the compressed representation, incorporating view-dependent geometric level-of-detail control. The surface light field representation can also be directly edited to yield plausible surface light fields for small changes in surface geometry and reflectance properties.

REFERENCES

• 1: D.I. Azuma. Interactive Rendering of Surface Light Fields. Technical Report UW-CSE-2000-04-01, Department of Computer Science and Engineering, University of Washington, April 2000.
• 2: Paul J. Besl , Neil D. McKay, A Method for Registration of 3-D Shapes, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.14 n.2, p.239-256, February 1992 [doi>10.1109/34.121791]
• 3: Brian Cabral , Marc Olano , Philip Nemec, Reflection space image based rendering, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.165-170, July 1999 [doi>10.1145/311535.311553]
• 4: Andrew Certain , Jovan Popovic , Tony DeRose , Tom Duchamp , David Salesin , Werner Stuetzle, Interactive multiresolution surface viewing, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.91-98, August 1996 [doi>10.1145/237170.237213]
• 5: Jonathan Cohen , Marc Olano , Dinesh Manocha, Appearance-preserving simplification, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.115-122, July 1998 [doi>10.1145/280814.280832]
• 6: Brian Curless , Marc Levoy, A volumetric method for building complex models from range images, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.303-312, August 1996 [doi>10.1145/237170.237269]
• 7: Paul E. Debevec , Camillo J. Taylor , Jitendra Malik, Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.11-20, August 1996 [doi>10.1145/237170.237191]
• 8: P. E. Debevec, Y. Yu, and G. D. Borshukov. Efficient View- Dependent Image-Based Rendering with Projective Texture-Mapping. Eurographics Rendering Workshop 1998, pages 105-116, June 1998.
• 9: Matthias Eck , Tony DeRose , Tom Duchamp , Hugues Hoppe , Michael Lounsbery , Werner Stuetzle, Multiresolution analysis of arbitrary meshes, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.173-182, September 1995 [doi>10.1145/218380.218440]
• 10: H. Gagnon, M. Soucy, R. Bergevin, and D. Laurendeau. Registration of Multiple Range Views for Automatic 3-D Model Building. In IEEE Conf. Computer Vision and Pattern Recognition, pages 581-586, June 1994.
• 11: Allen Gersho , Robert M. Gray, Vector quantization and signal compression, Kluwer Academic Publishers, Norwell, MA, 1991
• 12: Steven J. Gortler , Radek Grzeszczuk , Richard Szeliski , Michael F. Cohen, The lumigraph, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.43-54, August 1996 [doi>10.1145/237170.237200]
• 13: Wolfgang Heidrich , Hans-Peter Seidel, Realistic, hardware-accelerated shading and lighting, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.171-178, July 1999 [doi>10.1145/311535.311554]
• 14: Hugues Hoppe, Progressive meshes, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.99-108, August 1996 [doi>10.1145/237170.237216]
• 15: Hugues Hoppe, View-dependent refinement of progressive meshes, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.189-198, August 1997 [doi>10.1145/258734.258843]
• 16: Aaron W. F. Lee , Wim Sweldens , Peter Schröder , Lawrence Cowsar , David Dobkin, MAPS: multiresolution adaptive parameterization of surfaces, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.95-104, July 1998 [doi>10.1145/280814.280828]
• 17: Marc Levoy , Pat Hanrahan, Light field rendering, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.31-42, August 1996 [doi>10.1145/237170.237199]
• 18: Michael Lounsbery , Tony D. DeRose , Joe Warren, Multiresolution analysis for surfaces of arbitrary topological type, ACM Transactions on Graphics (TOG), v.16 n.1, p.34-73, Jan. 1997 [doi>10.1145/237748.237750]
• 19: David Luebke , Carl Erikson, View-dependent simplification of arbitrary polygonal environments, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.199-208, August 1997 [doi>10.1145/258734.258847]
• 20: M. Magnor and B. Girod. Adaptive Block-Based Light Field Coding. Proc. 3rd International Workshop on Synthetic and Natural Hybrid Coding and Three-Dimensional Imaging, pages 140-143, September 1999.
• 21: M. Magnor and B. Girod. Hierarchical Coding of Light Fields with Disparity Maps. Proc. IEEE International Conference on Image Processing, pages 334-338, October 1999.
• 22: G. S. P. Miller, S. Rubin, and D. Ponceleon. Lazy Decompression of Surface Light Fields for Precomputed Global Illumination. Eurographics Rendering Workshop 1998, pages 281-292, June 1998.
• 23: K. Nishino, Y. Sato, and K. Ikeuchi. Appearance compression and synthesis based on 3D model for mixed reality. In Proceedings of IEEE ICCV'99, pages 38 - 45, September 1999.
• 24: K. Nishino, Y. Sato, and K. Ikeuchi. Eigen-Texture Method: Appearance Compression based on 3D Model. Proc. of Computer Vision and Pattern Recognition, 1:618-624, June 1999.
• 25: K. Perlin , E. M. Hoffert, Hypertexture, ACM SIGGRAPH Computer Graphics, v.23 n.3, p.253-262, July 1989
• 26: Kari Pulli , Michael Cohen , Tom Duchamp , Hugues Hoppe , Linda G. Shapiro , Werner Stuetzle, View-base Rendering: Visualizing Real Objects from Scanned Range and Color Data, Proceedings of the Eurographics Workshop on Rendering Techniques '97, p.23-34, June 16-18, 1997
• 27: S.M. Rusinkiewicz. A New Change of Variables for Efficient BRDF Representation. In Eurographics Rendering Workshop 1998, pages 11-22. Eurographics, June 1998.
• 28: Yoichi Sato , Mark D. Wheeler , Katsushi Ikeuchi, Object shape and reflectance modeling from observation, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.379-387, August 1997 [doi>10.1145/258734.258885]
• 29: Gabriel Taubin, A signal processing approach to fair surface design, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.351-358, September 1995 [doi>10.1145/218380.218473]
• 30: R.Y. Tsai. An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pages 364-374, 1986.
• 31: Bruce Walter , Gün Alppay , Eric Lafortune , Sebastian Fernandez , Donald P. Greenberg, Fitting virtual lights for non-diffuse walkthroughs, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.45-48, August 1997 [doi>10.1145/258734.258766]
• 32: Julie C. Xia , Amitabh Varshney, Dynamic view-dependent simplification for polygonal models, Proceedings of the 7th conference on Visualization '96, p.327-ff., October 28-29, 1996, San Francisco, California, United States
• 33: Yizhou Yu , Paul Debevec , Jitendra Malik , Tim Hawkins, Inverse global illumination: recovering reflectance models of real scenes from photographs, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.215-224, July 1999 [doi>10.1145/311535.311559]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS

Additional Classification:
  G. Mathematics of Computing
  G.1 NUMERICAL ANALYSIS
      G.1.2 Approximation
          Subjects: Wavelets and fractals

  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.3 Picture/Image Generation
          Subjects: Digitizing and scanning; Viewing algorithms

General Terms:
Algorithms, Design, Measurement, Performance, Theory

Keywords:
3D photography, function quantization, image-based rendering, light field, lumigraph, principal function analysis, surface light fields, view-dependent level-of-detail, wavelets