4D frequency analysis of computational cameras for depth of field extension

ABSTRACT

Depth of field (DOF), the range of scene depths that appear sharp in a photograph, poses a fundamental tradeoff in photography---wide apertures are important to reduce imaging noise, but they also increase defocus blur. Recent advances in computational imaging modify the acquisition process to extend the DOF through deconvolution. Because deconvolution quality is a tight function of the frequency power spectrum of the defocus kernel, designs with high spectra are desirable. In this paper we study how to design effective extended-DOF systems, and show an upper bound on the maximal power spectrum that can be achieved. We analyze defocus kernels in the 4D light field space and show that in the frequency domain, only a low-dimensional 3D manifold contributes to focus. Thus, to maximize the defocus spectrum, imaging systems should concentrate their limited energy on this manifold. We review several computational imaging systems and show either that they spend energy outside the focal manifold or do not achieve a high spectrum over the DOF. Guided by this analysis we introduce the lattice-focal lens, which concentrates energy at the low-dimensional focal manifold and achieves a higher power spectrum than previous designs. We have built a prototype lattice-focal lens and present extended depth of field results.

REFERENCES

• 1: Adams, A., and Levoy, M. 2007. General linear cameras with finite aperture. In EGSR.
• 2: Aseem Agarwala , Mira Dontcheva , Maneesh Agrawala , Steven Drucker , Alex Colburn , Brian Curless , David Salesin , Michael Cohen, Interactive digital photomontage, ACM SIGGRAPH 2004 Papers, August 08-12, 2004, Los Angeles, California
• 3: Ben-Eliezer, E., Zalevsky, Z., Marom, E., and Konforti, N. 2005. Experimental realization of an imaging system with an extended depth of field. Applied Optics, 2792--2798.
• 4: Brenner, K., Lohmann, A., and Casteneda, J. O. 1983. The ambiguity function as a polar display of the OTF. Opt. Commun. 44, 323--326.
• 5: Dowski, E., and Cathey, W. 1995. Extended depth of field through wavefront coding. Applied Optics 34, 1859--1866.
• 6: FitzGerrell, A. R., Dowski, E., and Cathey, W. 1997. Defocus transfer function for circularly symmetric pupils. Applied Optics 36, 5796--5804.
• 7: George, N., and Chi, W. 2003. Computational imaging with the logarithmic asphere: theory. J. Opt. Soc. Am. A 20, 2260--2273.
• 8: Goodman, J. W. 1968. Introduction to Fourier Optics. McGraw-Hill Book Company.
• 9: Xianfeng Gu , Steven J. Gortler , Michael F. Cohen, Polyhedral Geometry and the Two-Plane Parameterization, Proceedings of the Eurographics Workshop on Rendering Techniques '97, p.1-12, June 16-18, 1997
• 10: Samuel W. Hasinoff , Kiriakos N. Kutulakos, Light-Efficient Photography, Proceedings of the 10th European Conference on Computer Vision: Part IV, October 12-18, 2008, Marseille, France [doi>10.1007/978-3-540-88693-8_4]
• 11: Hausler, G. 1972. A method to increase the depth of focus by two step image processing. Optics Communications, 3842.
• 12: Horn, B. K. P. 1968. Focusing. Tech. Rep. AIM-160, Massachusetts Institute of Technology.
• 13: Anat Levin , Rob Fergus , Frédo Durand , William T. Freeman, Image and depth from a conventional camera with a coded aperture, ACM SIGGRAPH 2007 papers, August 05-09, 2007, San Diego, California
• 14: Anat Levin , William T. Freeman , Frédo Durand, Understanding Camera Trade-Offs through a Bayesian Analysis of Light Field Projections, Proceedings of the 10th European Conference on Computer Vision: Part IV, October 12-18, 2008, Marseille, France [doi>10.1007/978-3-540-88693-8_7]
• 15: Levin, A., Freeman, W., and Durand, F. 2008. Understanding camera trade-offs through a Bayesian analysis of light field projections. MIT CSAIL TR 2008--049.
• 16: Anat Levin , Peter Sand , Taeg Sang Cho , Frédo Durand , William T. Freeman, Motion-invariant photography, ACM SIGGRAPH 2008 papers, August 11-15, 2008, Los Angeles, California
• 17: Levin, A., Hasinoff, S., Green, P., Durand, F., and Freeman, W. 2009. 4D frequency analysis of computational cameras for depth of field extension. MIT CSAIL TR 2009--019.
• 18: Levin, A., Weiss, Y., Durand, F., and Freeman, W. 2009. Understanding and evaluating blind deconvolution algorithms. In CVPR.
• 19: Marc Levoy , Pat Hanrahan, Light field rendering, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.31-42, August 1996 [doi>10.1145/237170.237199]
• 20: Hajime Nagahara , Sujit Kuthirummal , Changyin Zhou , Shree K. Nayar, Flexible Depth of Field Photography, Proceedings of the 10th European Conference on Computer Vision: Part IV, October 12-18, 2008, Marseille, France [doi>10.1007/978-3-540-88693-8_5]
• 21: Ren Ng, Fourier slice photography, ACM SIGGRAPH 2005 Papers, July 31-August 04, 2005, Los Angeles, California
• 22: Ogden, J., Adelson, E., Bergen, J. R., and Burt, P. 1985. Pyramid-based computer graphics. RCA Engineer 30, 5, 4--15.
• 23: Papoulis, A. 1974. Ambiguity function in fourier optics. Journal of the Optical Society of America A 64, 779--788.
• 24: Rihaczek, A. W. 1969. Principles of high-resolution radar. McGraw-Hill.
• 25: Ashok Veeraraghavan , Ramesh Raskar , Amit Agrawal , Ankit Mohan , Jack Tumblin, Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing, ACM SIGGRAPH 2007 papers, August 05-09, 2007, San Diego, California
• 26: Zhang, Z., and Levoy, M. 2009. Wigner distributions and how they relate to the light field. In ICCP.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.1 Digitization and Image Capture
      I.4.5 Reconstruction

General Terms:
Algorithms, Design, Measurement

Keywords:
Fourier analysis, computational camera, depth of field, light field