A photometric approach for estimating normals and tangents

ABSTRACT

This paper presents a technique for acquiring the shape of real-world objects with complex isotropic and anisotropic reflectance. Our method estimates the local normal and tangent vectors at each pixel in a reference view from a sequence of images taken under varying point lighting. We show that for many real-world materials and a restricted set of light positions, the 2D slice of the BRDF obtained by fixing the local view direction is symmetric under reflections of the halfway vector across the normal-tangent and normal-binormal planes. Based on this analysis, we develop an optimization that estimates the local surface frame by identifying these planes of symmetry in the measured BRDF. As with other photometric methods, a key benefit of our approach is that the input is easy to acquire and is less sensitive to calibration errors than stereo or multi-view techniques. Unlike prior work, our approach allows estimating the surface tangent in the case of anisotropic reflectance. We confirm the accuracy and reliability of our approach with analytic and measured data, present several normal and tangent fields acquired with our technique, and demonstrate applications to appearance editing.

REFERENCES

• 1: Alldrin, N. G., and Kriegman, D. J. 2007. Toward reconstructing surfaces with arbitrary isotropic reflectance: A stratified photometric stereo approach. In Proceedings of the International Conference on Computer Vision (ICCV), 1--8.
• 2: Alldrin, N., Zickler, T., and Kriegman, D. 2008. Photometric stereo with non-parametric and spatially-varying reflectance. In Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR).
• 3: Michael Ashikmin , Simon Premože , Peter Shirley, A microfacet-based BRDF generator, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.65-74, July 2000 [doi>10.1145/344779.344814]
• 4: James F. Blinn, Models of light reflection for computer synthesized pictures, Proceedings of the 4th annual conference on Computer graphics and interactive techniques, p.192-198, July 20-22, 1977, San Jose, California
• 5: Brian Cabral , Marc Olano , Philip Nemec, Reflection space image based rendering, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.165-170, July 1999 [doi>10.1145/311535.311553]
• 6: Tongbo Chen , Michael Goesele , Hans-Peter Seidel, Mesostructure from Specularity, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.1825-1832, June 17-22, 2006 [doi>10.1109/CVPR.2006.182]
• 7: Chung, H.-S., and Jia, J. 2008. Efficient photometric stereo on glossy surfaces with wide specular lobes. In Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR).
• 8: Coleman, E., and Jain, R. 1982. Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry. Computer Vision, Graphics and Image Processing 18, 4, 309--328.
• 9: R. L. Cook , K. E. Torrance, A Reflectance Model for Computer Graphics, ACM Transactions on Graphics (TOG), v.1 n.1, p.7-24, Jan. 1982 [doi>10.1145/357290.357293]
• 10: Matthew Fisher , Peter Schröder , Mathieu Desbrun , Hugues Hoppe, Design of tangent vector fields, ACM Transactions on Graphics (TOG), v.26 n.3, July 2007
• 11: Francken, Y., Cuypers, T., Mertens, T., Gielis, J., and Bekaert, P. 2008. High quality mesostructure acquisition using specularities. In Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR).
• 12: Athinodoros S. Georghiades, Incorporating the Torrance and Sparrow Model of Reflectance in Uncalibrated Photometric Stereo, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.816, October 13-16, 2003
• 13: Dan B. Goldman , Brian Curless , Aaron Hertzmann , Steven M. Seitz, Shape and Spatially-Varying BRDFs from Photometric Stereo, Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, p.341-348, October 17-20, 2005 [doi>10.1109/ICCV.2005.219]
• 14: Xiao D. He , Kenneth E. Torrance , François X. Sillion , Donald P. Greenberg, A comprehensive physical model for light reflection, Proceedings of the 18th annual conference on Computer graphics and interactive techniques, p.175-186, July 1991
• 15: Wolfgang Heidrich , Hans-Peter Seidel, Realistic, hardware-accelerated shading and lighting, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.171-178, July 1999 [doi>10.1145/311535.311554]
• 16: Hertzmann, A., and Seitz, S. 2003. Shape and materials by example: a photometric stereo approach. IEEE Computer Vision and Pattern Recognition, 2003. 1, 533--540.
• 17: Jason Lawrence , Aner Ben-Artzi , Christopher DeCoro , Wojciech Matusik , Hanspeter Pfister , Ravi Ramamoorthi , Szymon Rusinkiewicz, Inverse shade trees for non-parametric material representation and editing, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
• 18: Ma, W.-C., Hawkins, T., Peers, P., Chabert, C.-F., Weiss, M., and Debevec, P. 2007. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. In Proceedings of Eurographics Symposium on Rendering.
• 19: Satya P. Mallick , Todd E. Zickler , David J. Kriegman , Peter N. Belhumeur, Beyond Lambert: Reconstructing Specular Surfaces Using Color, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2, p.619-626, June 20-26, 2005 [doi>10.1109/CVPR.2005.88]
• 20: Vincent Masselus , Philip Dutré , Frederik Anrys, The free-form light stage, Proceedings of the 13th Eurographics workshop on Rendering, June 26-28, 2002, Pisa, Italy
• 21: Nayar, S., Ikeuchi, K., and Kanade, T. 1990. Determining shape and reflectance of hybrid surfaces by photometricsampling. Robotics and Automation, IEEE Transactions on 6, 4, 418--431.
• 22: Shree K. Nayar , Gurunandan Krishnan , Michael D. Grossberg , Ramesh Raskar, Fast separation of direct and global components of a scene using high frequency illumination, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
• 23: Nelder, J., and Mead, R. 1965. A simplex method for function minimization. Computer Journal 7, 308--311.
• 24: Ngan, A., Durand, F., and Matusik, W. 2005. Experimental analysis of BRDF models. In Proceedings of the Eurographics Symposium on Rendering, 117--126.
• 25: Nicodemus, F. E., Richmond, J. C., and Hsia, J. J. 1977. Geometrical considerations and reflectance. National Bureau of Standards.
• 26: Joshua Podolak , Philip Shilane , Aleksey Golovinskiy , Szymon Rusinkiewicz , Thomas Funkhouser, A planar-reflective symmetry transform for 3D shapes, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
• 27: Rusinkiewicz, S. 1998. A new change of variables for efficient BRDF representation. In Proceedings of the Eurugraphics Rendering Workshop, 11--22.
• 28: Schlick, C. 1994. An inexpensive BRDF model for physically-based rendering. Computer Graphics Forum 13, 3, 233--246.
• 29: Jonathan Richard Shewchuk, Triangle: Engineering a 2D Quality Mesh Generator and Delaunay Triangulator, Selected papers from the Workshop on Applied Computational Geormetry, Towards Geometric Engineering, p.203-222, May 27-28, 1996
• 30: Stanford, 2002. Stanford graphics lab spherical gantry (http://graphics.stanford.edu/projects/gantry/).
• 31: Michael M. Stark , James Arvo , Brian Smits, Barycentric Parameterizations for Isotropic BRDFs, IEEE Transactions on Visualization and Computer Graphics, v.11 n.2, p.126-138, March 2005 [doi>10.1109/TVCG.2005.26]
• 32: Hemant D. Tagare , Rui J. P. deFigueiredo, A Theory of Photometric Stereo for a Class of Diffuse Non-Lambertian Surfaces, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.13 n.2, p.133-152, February 1991 [doi>10.1109/34.67643]
• 33: Corey Toler-Franklin , Adam Finkelstein , Szymon Rusinkiewicz, Illustration of complex real-world objects using images with normals, Proceedings of the 5th international symposium on Non-photorealistic animation and rendering, August 04-05, 2007, San Diego, California [doi>10.1145/1274871.1274889]
• 34: Torrance, K. E., and Sparrow, M. E. 1967. Theory for off-specular reflection from roughened surfaces. Journal of the Optical Society of America 57, 1105--1114.
• 35: Jing Wang , Kristin J. Dana, Relief Texture from Specularities, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.28 n.3, p.446-457, March 2006 [doi>10.1109/TPAMI.2006.63]
• 36: Gregory J. Ward, Measuring and modeling anisotropic reflection, Proceedings of the 19th annual conference on Computer graphics and interactive techniques, p.265-272, July 1992
• 37: Woodham, R. J. 1980. Photometric method for determining surface orientation from multiple images. Optical Engineering 19, 1, 139--144.
• 38: Todd E. Zickler , Peter N. Belhumeur , David J. Kriegman, Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction, International Journal of Computer Vision, v.49 n.2-3, p.215-227, September-October 2002 [doi>10.1023/A:1020149707513]

INDEX TERMS

Keywords:
BRDF, anisotropic, normal map, photometric stereo, symmetry, tangent map