Digital photography with flash and no-flash image pairs

ABSTRACT

Digital photography has made it possible to quickly and easily take a pair of images of low-light environments: one with flash to capture detail and one without flash to capture ambient illumination. We present a variety of applications that analyze and combine the strengths of such flash/no-flash image pairs. Our applications include denoising and detail transfer (to merge the ambient qualities of the no-flash image with the high-frequency flash detail), white-balancing (to change the color tone of the ambient image), continuous flash (to interactively adjust flash intensity), and red-eye removal (to repair artifacts in the flash image). We demonstrate how these applications can synthesize new images that are of higher quality than either of the originals.

REFERENCES

• 1: Jim Adams , Ken Parulski , Kevin Spaulding, Color Processing in Digital Cameras, IEEE Micro, v.18 n.6, p.20-30, November 1998 [doi>10.1109/40.743681]
• 2: Aseem Agarwala , Mira Dontcheva , Maneesh Agrawala , Steven Drucker , Alex Colburn , Brian Curless , David Salesin , Michael Cohen, Interactive digital photomontage, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004
• 3: David Akers , Frank Losasso , Jeff Klingner , Maneesh Agrawala , John Rick , Pat Hanrahan, Conveying Shape and Features with Image-Based Relighting, Proceedings of the 14th IEEE Visualization 2003 (VIS'03), p.46, October 22-24, 2003 [doi>10.1109/VISUAL.2003.1250392]
• 4: Prasun Choudhury , Jack Tumblin, The trilateral filter for high contrast images and meshes, Proceedings of the 14th Eurographics workshop on Rendering, June 25-27, 2003, Leuven, Belgium
• 5: Paul E. Debevec , Jitendra Malik, Recovering high dynamic range radiance maps from photographs, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.369-378, August 1997 [doi>10.1145/258734.258884]
• 6: Paul Debevec , Tim Hawkins , Chris Tchou , Haarm-Pieter Duiker , Westley Sarokin , Mark Sagar, Acquiring the reflectance field of a human face, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.145-156, July 2000 [doi>10.1145/344779.344855]
• 7: DICARLO, J. M., XIAO, F., AND WANDELL, B. A., 2001. Illuminating illumination. Ninth Color Imaging Conference, pp. 27--34.
• 8: Frédo Durand , Julie Dorsey, Fast bilateral filtering for the display of high-dynamic-range images, ACM Transactions on Graphics (TOG), v.21 n.3, July 2002
• 9: Elmar Eisemann , Frédo Durand, Flash photography enhancement via intrinsic relighting, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004
• 10: Shachar Fleishman , Iddo Drori , Daniel Cohen-Or, Bilateral mesh denoising, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 11: G. Funka-Lea , R. Bajcsy, Combining color and geometry for the active, visual recognition of shadows, Proceedings of the Fifth International Conference on Computer Vision, p.203, June 20-23, 1995
• 12: GAUBATZ, M., AND ULICHNEY, R., 2002. Automatic red-eye detection and correction. IEEE International Conference on Image Processing, pp. 804--807.
• 13: HAEBERLI, P., 1992. Synthetic lighting for photography. Grafica Obscura, http://www.sgi.com/grafica/synth.
• 14: JIA, J., SUN, J., TANG, C.-K., AND SHUM, H., 2004. Bayesian correction of image intensity with spatial consideration. ECCV 2004, LNCS 3023, pp. 342--354.
• 15: Thouis R. Jones , Frédo Durand , Mathieu Desbrun, Non-iterative, feature-preserving mesh smoothing, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 16: Sing Bing Kang , Matthew Uyttendaele , Simon Winder , Richard Szeliski, High dynamic range video, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 17: KODAK, 2001. CCD Image Sensor Noise Sources. Application Note MPT/PS-0233.
• 18: Sang Wook Lee , Ruzena Bajcsy, Detection of Specularity Using Color and Multiple Views, Proceedings of the Second European Conference on Computer Vision, p.99-114, May 19-22, 1992
• 19: Zicheng Liu , Ying Shan , Zhengyou Zhang, Expressive expression mapping with ratio images, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.271-276, August 2001 [doi>10.1145/383259.383289]
• 20: Vincent Masselus , Philip Dutré , Frederik Anrys, The free-form light stage, Proceedings of the 13th Eurographics workshop on Rendering, June 26-28, 2002, Pisa, Italy
• 21: Byong Mok Oh , Max Chen , Julie Dorsey , Frédo Durand, Image-based modeling and photo editing, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.433-442, August 2001 [doi>10.1145/383259.383310]
• 22: PATTI, A., KONSTANTINIDES, K., TRETTER, D. AND LIN, Q., 1998. Automatic digital redeye reduction. IEEE International Conference on Image Processing, pp. 55--59.
• 23: P. Perona , J. Malik, Scale-Space and Edge Detection Using Anisotropic Diffusion, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.12 n.7, p.629-639, July 1990 [doi>10.1109/34.56205]
• 24: Ramesh Raskar , Jingyi Yu , Adrian Ilie, A Non-Photorealistic Camera: Detecting Silhouettes With Multi-Flash: A method for capturing scenes that pose a problem for traditional cameras (low-contrast, geometrically complex, or action scenes). The resulting images are simpler and easier to understand for illustrating complex mechanical parts, image simplification, image resizing, non-photorealistic rendering, and video surveillance, Proceedings of the SIGGRAPH 2003 conference on Sketches & applications: in conjunction with the 30th annual conference on Computer graphics and interactive techniques, July 27-31, 2003, San Diego, California [doi>10.1145/965400.1077121]
• 25: SCHROEDER, M., AND MOSER, S., 2001. Automatic color correction based on generic content-based image analysis. Ninth Color Imaging Conference, pp. 41--45.
• 26: Amnon Shashua , Tammy Riklin-Raviv, The Quotient Image: Class-Based Re-Rendering and Recognition with Varying Illuminations, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.2, p.129-139, February 2001 [doi>10.1109/34.908964]
• 27: Rahul Swaminathan , Shree B. Kang , Richard Szeliski , Antonio Criminisi , Shree K. Nayar, On the Motion and Appearance of Specularities in Image Sequences, Proceedings of the 7th European Conference on Computer Vision-Part I, p.508-523, May 28-31, 2002
• 28: Richard Szeliski , Heung-Yeung Shum, Creating full view panoramic image mosaics and environment maps, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.251-258, August 1997 [doi>10.1145/258734.258861]
• 29: C. Tomasi , R. Manduchi, Bilateral Filtering for Gray and Color Images, Proceedings of the Sixth International Conference on Computer Vision, p.839, January 04-07, 1998

INDEX TERMS

Keywords:
Noise removal, bilateral filtering, detail transfer, flash photography, image fusion, image processing, red-eye removal, sharpening, white balancing