Dual photography

ABSTRACT

We present a novel photographic technique called dual photography, which exploits Helmholtz reciprocity to interchange the lights and cameras in a scene. With a video projector providing structured illumination, reciprocity permits us to generate pictures from the viewpoint of the projector, even though no camera was present at that location. The technique is completely image-based, requiring no knowledge of scene geometry or surface properties, and by its nature automatically includes all transport paths, including shadows, inter-reflections and caustics. In its simplest form, the technique can be used to take photographs without a camera; we demonstrate this by capturing a photograph using a projector and a photo-resistor. If the photo-resistor is replaced by a camera, we can produce a 4D dataset that allows for relighting with 2D incident illumination. Using an array of cameras we can produce a 6D slice of the 8D reflectance field that allows for relighting with arbitrary light fields. Since an array of cameras can operate in parallel without interference, whereas an array of light sources cannot, dual photography is fundamentally a more efficient way to capture such a 6D dataset than a system based on multiple projectors and one camera. As an example, we show how dual photography can be used to capture and relight scenes.

REFERENCES

• 1: Paul E. Debevec , Jitendra Malik, Recovering high dynamic range radiance maps from photographs, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.369-378, August 1997 [doi>10.1145/258734.258884]
• 2: Paul Debevec , Tim Hawkins , Chris Tchou , Haarm-Pieter Duiker , Westley Sarokin , Mark Sagar, Acquiring the reflectance field of a human face, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.145-156, July 2000 [doi>10.1145/344779.344855]
• 3: Goesele, M., Lensch, H. P. A., Lang, J., Fuchs, C., and Seidel, H.-P. 2004. DISCO: Acquisition of Translucent Objects. In SIGGRAPH '04, 835--844.
• 4: Steven J. Gortler , Radek Grzeszczuk , Richard Szeliski , Michael F. Cohen, The lumigraph, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.43-54, August 1996 [doi>10.1145/237170.237200]
• 5: Marc Levoy , Pat Hanrahan, Light field rendering, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.31-42, August 1996 [doi>10.1145/237170.237199]
• 6: Levoy, M., Chen, B., Vaish, V., Horowitz, M., McDowall, I., and Bolas, M. 2004. Synthetic Aperture Confocal Imaging. In SIGGRAPH '04, 825--834.
• 7: Masselus, V., Peers, P., Dutré, P., and Willems, Y. D. 2003. Relighting with 4D Incident Light Fields. In SIGGRAPH '03, 613--620.
• 8: Wojciech Matusik , Hanspeter Pfister , Addy Ngan , Paul Beardsley , Remo Ziegler , Leonard McMillan, Image-based 3D photography using opacity hulls, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 9: Wojciech Matusik , Hanspeter Pfister , Remo Ziegler , Addy Ngan , Leonard McMillan, Acquisition and rendering of transparent and refractive objects, Proceedings of the 13th Eurographics workshop on Rendering, June 26-28, 2002, Pisa, Italy
• 10: Matusik, W., Loper, M., and Pfister, H. 2004. Progressively-Refined Reflectance Functions for Natural Illumination. In Eurographics Symposium on Rendering, 299--308.
• 11: Pieter Peers , Philip Dutré, Wavelet environment matting, Proceedings of the 14th Eurographics workshop on Rendering, June 25-27, 2003, Leuven, Belgium
• 12: Rayleigh, J. W. S. B. 1900. On the Law of Reciprocity in Diffuse Reflexion. Philosophical Magazine 49, 324--325.
• 13: Robertson, M. A., S., and Stevenson, R. L. 1999. Dynamic Range Improvement through Multiple Exposures. In Proceedings of the IEEE Intl. Conference on Image Processing (ICIP'99), 159--163.
• 14: Szymon Rusinkiewicz , Olaf Hall-Holt , Marc Levoy, Real-time 3D model acquisition, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 15: Yoav Y. Schechner , Shree K. Nayar , Peter N. Belhumeur, A Theory of Multiplexed Illumination, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.808, October 13-16, 2003
• 16: Tu, P., Mendonca, P., Ross, J., and Miller, J. 2003. Surface Registration with a Helmholtz Reciprocity Image Pair. In IEEE Workshop on Color and Photometric Methods in Computer Vision.
• 17: Eric Veach , Leonidas J. Guibas, Robust monte carlo methods for light transport simulation, 1998
• 18: Von Helmholtz, H. 1856. Treatise on Physiological Optics (1925). The Optical Society of America. Electronic edition (2001): University of Pennsylvania http://psych.upenn.edu/backuslab/helmholtz.
• 19: Turner Whitted, An improved illumination model for shaded display, Communications of the ACM, v.23 n.6, p.343-349, June 1980 [doi>10.1145/358876.358882]
• 20: Wilburn, B., Joshi, N., Vaish, V., Talvala, E.-V., Antunez, E., Barth, A., Adams, A., Levoy, M., and Horowitz, M. 2005. High Performance Imaging using Large Camera Arrays. In SIGGRAPH '05.
• 21: Todd E. Zickler , Peter N. Belhumeur , David J. Kriegman, Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction, International Journal of Computer Vision, v.49 n.2-3, p.215-227, September-October 2002 [doi>10.1023/A:1020149707513]
• 22: Douglas E. Zongker , Dawn M. Werner , Brian Curless , David H. Salesin, Environment matting and compositing, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.205-214, July 1999 [doi>10.1145/311535.311558]
• 23: Zotkin, D., Duraiswami, R., Grassi, E., and Gumerov, N. 2004. Fast Head Related Transfer Function Measurement via Reciprocity. Technical Report No. CS-4620, University of Maryland.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.3 Picture/Image Generation
          Subjects: Digitizing and scanning

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Color, shading, shadowing, and texture

Keywords:
dual photography, image synthesis, image-based relighting