Fast separation of direct and global components of a scene using high frequency illumination

ABSTRACT

We present fast methods for separating the direct and global illumination components of a scene measured by a camera and illuminated by a light source. In theory, the separation can be done with just two images taken with a high frequency binary illumination pattern and its complement. In practice, a larger number of images are used to overcome the optical and resolution limitations of the camera and the source. The approach does not require the material properties of objects and media in the scene to be known. However, we require that the illumination frequency is high enough to adequately sample the global components received by scene points. We present separation results for scenes that include complex interreflections, subsurface scattering and volumetric scattering. Several variants of the separation approach are also described. When a sinusoidal illumination pattern is used with different phase shifts, the separation can be done using just three images. When the computed images are of lower resolution than the source and the camera, smoothness constraints are used to perform the separation using a single image. Finally, in the case of a static scene that is lit by a simple point source, such as the sun, a moving occluder and a video camera can be used to do the separation. We also show several simple examples of how novel images of a scene can be computed from the separation results.

REFERENCES

• 1: Okan Arikan , David A. Forsyth , James F. O'Brien, Fast and detailed approximate global illumination by irradiance decomposition, ACM Transactions on Graphics (TOG), v.24 n.3, July 2005
• 2: Jean-Yves Bouguet , Pietro Perona, 3D Photography Using Shadows in Dual-Space Geometry, International Journal of Computer Vision, v.35 n.2, p.129-149, Nov.-Dec. 1999 [doi>10.1023/A:1008124523456]
• 3: Chandrasekhar, S. 1950. Radiative Transfer. Clarendon Press, Oxford, UK. reprinted by Dover Publications, 1960.
• 4: Yung-Yu Chuang , Douglas E. Zongker , Joel Hindorff , Brian Curless , David H. Salesin , Richard Szeliski, Environment matting extensions: towards higher accuracy and real-time capture, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.121-130, July 2000 [doi>10.1145/344779.344844]
• 5: Paul Debevec , Tim Hawkins , Chris Tchou , Haarm-Pieter Duiker , Westley Sarokin , Mark Sagar, Acquiring the reflectance field of a human face, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.145-156, July 2000 [doi>10.1145/344779.344855]
• 6: David Forsyth , Andrew Zisserman, Reflections on Shading, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.13 n.7, p.671-679, July 1991 [doi>10.1109/34.85657]
• 7: Steven J. Gortler , Radek Grzeszczuk , Richard Szeliski , Michael F. Cohen, The lumigraph, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.43-54, August 1996 [doi>10.1145/237170.237200]
• 8: David S. Immel , Michael F. Cohen , Donald P. Greenberg, A radiosity method for non-diffuse environments, Proceedings of the 13th annual conference on Computer graphics and interactive techniques, p.133-142, August 1986
• 9: James T. Kajiya, The rendering equation, Proceedings of the 13th annual conference on Computer graphics and interactive techniques, p.143-150, August 1986
• 10: Koenderink, J., and Van Doorn, A. 1983. Geometrical modes as a general method to treat diffuse interreflections in radiometry. JOSA 73, 6 (June), 843--850.
• 11: Marc Levoy , Pat Hanrahan, Light field rendering, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.31-42, August 1996 [doi>10.1145/237170.237199]
• 12: Zhouchen Lin , Tien-Tsin Wong , Heung-Yeung Shum, Relighting with the Reflected Irradiance Field: Representation, Sampling and Reconstruction, International Journal of Computer Vision, v.49 n.2-3, p.229-246, September-October 2002 [doi>10.1023/A:1020153824351]
• 13: Mallick, S. P., Zickler, T., Belhumeur, P. N., and Kriegman, D. J. 2006. Specularity removal in images and videos: A PDE approach. In Proc. of ECCV.
• 14: Shree K. Nayar , Katsushi Ikeuchi , Takeo Kanade, Shape from interreflections, International Journal of Computer Vision, v.6 n.3, p.173-195, Aug. 1991 [doi>10.1007/BF00115695]
• 15: Pieter Peers , Philip Dutré, Wavelet environment matting, Proceedings of the 14th Eurographics workshop on Rendering, June 25-27, 2003, Leuven, Belgium
• 16: Steven M. Seitz , Yasuyuki Matsushita , Kiriakos N. Kutulakos, A Theory of Inverse Light Transport, Proceedings of the Tenth IEEE International Conference on Computer Vision, p.1440-1447, October 17-20, 2005 [doi>10.1109/ICCV.2005.25]
• 17: Pradeep Sen , Billy Chen , Gaurav Garg , Stephen R. Marschner , Mark Horowitz , Marc Levoy , Hendrik P. A. Lensch, Dual photography, ACM Transactions on Graphics (TOG), v.24 n.3, July 2005
• 18: Shim, H., and Chen, T. 2005. A statistical framework for image-based relighting. In Proc. of ICASSP.
• 19: Peter-Pike Sloan , Jan Kautz , John Snyder, Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lighting environments, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 20: Tsumura, N., Ojima, N., Sato, K., Shiraishi, M., Shimizu, H., Nabeshima, H., Akazaki, S., Hori, K., and Miyake, Y. 2003. Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin. In Proc. of SIGGRAPH, ACM Press, 770--779.
• 21: Woodham, R. 1980. Photometric Method for Determining Surface Orientation from Multiple Images. Optical Engineering 19, 1 (January), 139--144.
• 22: Clarence Wust , David W. Capson, Surface profile measurement using color fringe projection, Machine Vision and Applications, v.4 n.3, p.193-203, Summer 1991 [doi>10.1007/BF01230201]
• 23: Jiayuan Zhu , Yee-Hong Yang, Frequency-Based Environment Matting, Proceedings of the Computer Graphics and Applications, 12th Pacific Conference on (PG'04), p.402-410, October 06-08, 2004
• 24: Douglas E. Zongker , Dawn M. Werner , Brian Curless , David H. Salesin, Environment matting and compositing, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.205-214, July 1999 [doi>10.1145/311535.311558]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Color, shading, shadowing, and texture

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.1 Digitization and Image Capture
          Subjects: Radiometry

Keywords:
coded illumination, direct illumination, global illumination, image decomposition, image manipulation, interreflections, subsurface scattering, translucency, volumetric scattering