Shield fields: modeling and capturing 3D occluders

ABSTRACT

We describe a unified representation of occluders in light transport and photography using shield fields: the 4D attenuation function which acts on any light field incident on an occluder. Our key theoretical result is that shield fields can be used to decouple the effects of occluders and incident illumination. We first describe the properties of shield fields in the frequency-domain and briefly analyze the "forward" problem of efficiently computing cast shadows. Afterwards, we apply the shield field signal-processing framework to make several new observations regarding the "inverse" problem of reconstructing 3D occluders from cast shadows -- extending previous work on shape-from-silhouette and visual hull methods. From this analysis we develop the first single-camera, single-shot approach to capture visual hulls without requiring moving or programmable illumination. We analyze several competing camera designs, ultimately leading to the development of a new large-format, mask-based light field camera that exploits optimal tiled-broadband codes for light-efficient shield field capture. We conclude by presenting a detailed experimental analysis of shield field capture and 3D occluder reconstruction.

REFERENCES

• 1: Accorsia, R., Gasparini, F., and Lanza, R. C. 2001. Optimal coded aperture patterns for improved SNR in nuclear medicine imaging. Nuclear Instruments and Methods in Physics Research A 474, 3, 273--284.
• 2: Edward H. Adelson , John Y. A. Wang, Single Lens Stereo with a Plenoptic Camera, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.14 n.2, p.99-106, February 1992 [doi>10.1109/34.121783]
• 3: BowHaus, Inc., 2007. BowHaus LVT specs and resolutions. http://www.bowhaus.com/services/lvtspecs.php4.
• 4: Jin-Xiang Chai , Xin Tong , Shing-Chow Chan , Heung-Yeung Shum, Plenoptic sampling, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.307-318, July 2000 [doi>10.1145/344779.344932]
• 5: Paul Debevec , Tim Hawkins , Chris Tchou , Haarm-Pieter Duiker , Westley Sarokin , Mark Sagar, Acquiring the reflectance field of a human face, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.145-156, July 2000 [doi>10.1145/344779.344855]
• 6: Frédo Durand , Nicolas Holzschuch , Cyril Soler , Eric Chan , François X. Sillion, A frequency analysis of light transport, ACM Transactions on Graphics (TOG), v.24 n.3, July 2005
• 7: Hany Samir Farid, Range estimation by optical differentiation, University of Pennsylvania, Philadelphia, PA, 1997
• 8: Fenimore, E., and Cannon, T. 1978. Coded aperture imaging with uniformly redundant arrays. Appl. Optics 17, 3, 337--347.
• 9: Georgiev, T., Zheng, K. C., Curless, B., Salesin, D., Na-yar, S., and Intwala, C. 2006. Spatio-angular resolution tradeoffs in integral photography. In EGSR, 263--272.
• 10: Goodman, J. W. 1996. Introduction to Fourier Optics.
• 11: Steven J. Gortler , Radek Grzeszczuk , Richard Szeliski , Michael F. Cohen, The lumigraph, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.43-54, August 1996 [doi>10.1145/237170.237200]
• 12: Markus Gross , Stephan Würmlin , Martin Naef , Edouard Lamboray , Christian Spagno , Andreas Kunz , Esther Koller-Meier , Tomas Svoboda , Luc Van Gool , Silke Lang , Kai Strehlke , Andrew Vande Moere , Oliver Staadt, blue-c: a spatially immersive display and 3D video portal for telepresence, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 13: Aaron Isaksen , Leonard McMillan , Steven J. Gortler, Dynamically reparameterized light fields, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.297-306, July 2000 [doi>10.1145/344779.344929]
• 14: Ives, H. E. 1928. Camera for making parallax panoramagrams. J. Opt. Soc. of America 17, 435--439.
• 15: Anat Levin , Rob Fergus , Frédo Durand , William T. Freeman, Image and depth from a conventional camera with a coded aperture, ACM Transactions on Graphics (TOG), v.26 n.3, July 2007
• 16: Marc Levoy , Pat Hanrahan, Light field rendering, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.31-42, August 1996 [doi>10.1145/237170.237199]
• 17: Lippmann, G. 1908. Epreuves reversible donnant la sensation du relief. Journal of Physics 7, 4, 821--825.
• 18: Wojciech Matusik , Chris Buehler , Leonard McMillan, Polyhedral Visual Hulls for Real-Time Rendering, Proceedings of the 12th Eurographics Workshop on Rendering Techniques, p.115-126, June 25-27, 2001
• 19: Shree K. Nayar , Gurunandan Krishnan , Michael D. Grossberg , Ramesh Raskar, Fast separation of direct and global components of a scene using high frequency illumination, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
• 20: Ng, R., Levoy, M., Brédif, M., Duval, G., Horowitz, M., and Hanrahan, P. 2005. Light field photography with a hand-held plenoptic camera. Tech. rep., Stanford University.
• 21: Ren Ng, Fourier slice photography, ACM Transactions on Graphics (TOG), v.24 n.3, July 2005
• 22: Ravi Ramamoorthi , Melissa Koudelka , Peter Belhumeur, A Fourier Theory for Cast Shadows, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.2, p.288-295, February 2005 [doi>10.1109/TPAMI.2005.22]
• 23: Ravi Ramamoorthi , Dhruv Mahajan , Peter Belhumeur, A first-order analysis of lighting, shading, and shadows, ACM Transactions on Graphics (TOG), v.26 n.1, p.2-es, January 2007 [doi>10.1145/1189762.1189764]
• 24: Ramesh Raskar , Amit Agrawal , Jack Tumblin, Coded exposure photography: motion deblurring using fluttered shutter, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
• 25: Savarese, S., Rushmeier, H., Bernardini, F., and Per-ona, P. 2001. Shadow carving. In ICCV, 190--197.
• 26: Cyril Soler , François X. Sillion, Fast calculation of soft shadow textures using convolution, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.321-332, July 1998 [doi>10.1145/280814.280927]
• 27: Eino-Ville Talvala , Andrew Adams , Mark Horowitz , Marc Levoy, Veiling glare in high dynamic range imaging, ACM Transactions on Graphics (TOG), v.26 n.3, July 2007
• 28: Thornber, K., and Jacobs, D. 2001. Cast shadows and linear subspaces. Tech. rep. TR-2001-100, NEC.
• 29: Ashok Veeraraghavan , Ramesh Raskar , Amit Agrawal , Ankit Mohan , Jack Tumblin, Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing, ACM Transactions on Graphics (TOG), v.26 n.3, July 2007
• 30: Daniel Vlasic , Hanspeter Pfister , Sergey Molinov , Radek Grzeszczuk , Wojciech Matusik, Opacity light fields: interactive rendering of surface light fields with view-dependent opacity, Proceedings of the 2003 symposium on Interactive 3D graphics, April 27-30, 2003, Monterey, California [doi>10.1145/641480.641496]
• 31: Bennett Wilburn , Neel Joshi , Vaibhav Vaish , Eino-Ville Talvala , Emilio Antunez , Adam Barth , Andrew Adams , Mark Horowitz , Marc Levoy, High performance imaging using large camera arrays, ACM Transactions on Graphics (TOG), v.24 n.3, July 2005
• 32: Daniel N. Wood , Daniel I. Azuma , Ken Aldinger , Brian Curless , Tom Duchamp , David H. Salesin , Werner Stuetzle, Surface light fields for 3D photography, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.287-296, July 2000 [doi>10.1145/344779.344925]
• 33: Yamazaki, S., Narasimhan, S. G., Baker, S., and Kanade, T. 2007. Coplanar shadowgrams for acquiring visual hulls of intricate objects. In ICCV, 1--8.
• 34: Kun Zhou , Yaohua Hu , Stephen Lin , Baining Guo , Heung-Yeung Shum, Precomputed shadow fields for dynamic scenes, ACM Transactions on Graphics (TOG), v.24 n.3, July 2005
• 35: Zwicker, M., Matusik, W., Durand, F., and Pfister, H. 2006. Antialiasing for automultiscopic displays. In EGSR, 1--10.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.3 Picture/Image Generation
          Subjects: Digitizing and scanning

Keywords:
cast shadows, coded aperture imaging, computational photography, light fields, light transport, visual hull