Synthesizing bidirectional texture functions for real-world surfaces

ABSTRACT

In this paper, we present a novel approach to synthetically generating bidirectional texture functions (BTFs) of real-world surfaces. Unlike a conventional two-dimensional texture, a BTF is a six-dimensional function that describes the appearance of texture as a function of illumination and viewing directions. The BTF captures the appearance change caused by visible small-scale geometric details on surfaces. From a sparse set of images under different viewing/lighting settings, our approach generates BTFs in three steps. First, it recovers approximate 3D geometry of surface details using a shape-from-shading method. Then, it generates a novel version of the geometric details that has the same statistical properties as the sample surface with a non-parametric sampling method. Finally, it employs an appearance preserving procedure to synthesize novel images for the recovered or generated geometric details under various viewing/lighting settings, which then define a BTF. Our experimental results demonstrate the effectiveness of our approach.

REFERENCES

• 1: E.H. Adelson and J.R. Bergen. The plenoptic function and the elements of early vision. In Computational Models of Visual Processing, pages 3-20. MIT Press, Cambridge, Mass., 1991.
• 2: James F. Blinn, Models of light reflection for computer synthesized pictures, Proceedings of the 4th annual conference on Computer graphics and interactive techniques, p.192-198, July 20-22, 1977, San Jose, California
• 3: James F. Blinn, Simulation of wrinkled surfaces, Proceedings of the 5th annual conference on Computer graphics and interactive techniques, p.286-292, August 23-25, 1978
• 4: Jeremy S. De Bonet, Multiresolution sampling procedure for analysis and synthesis of texture images, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.361-368, August 1997 [doi>10.1145/258734.258882]
• 5: M.J. Brooks and B.K.P. Horn. Shape and source from shading. In Proc. Intern. Joint Conf. Art. Int., pages 932-936, 1988.
• 6: Edwin Earl Catmull, A subdivision algorithm for computer display of curved surfaces., 1974
• 7: Kristin J. Dana , Bram van Ginneken , Shree K. Nayar , Jan J. Koenderink, Reflectance and texture of real-world surfaces, ACM Transactions on Graphics (TOG), v.18 n.1, p.1-34, Jan. 1999 [doi>10.1145/300776.300778]
• 8: K. J. Dana , S. K. Nayar, Histogram Model for 3D Textures, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.618, June 23-25, 1998
• 9: Kristin J. Dana , Shree K. Nayar, Correlation Model for 3D Texture, Proceedings of the International Conference on Computer Vision-Volume 2, p.1061, September 20-25, 1999
• 10: Paul Debevec , Tim Hawkins , Chris Tchou , Haarm-Pieter Duiker , Westley Sarokin , Mark Sagar, Acquiring the reflectance field of a human face, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.145-156, July 2000 [doi>10.1145/344779.344855]
• 11: Paul E. Debevec , Camillo J. Taylor , Jitendra Malik, Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.11-20, August 1996 [doi>10.1145/237170.237191]
• 12: Alexei A. Efros , Thomas K. Leung, Texture Synthesis by Non-Parametric Sampling, Proceedings of the International Conference on Computer Vision-Volume 2, p.1033, September 20-25, 1999
• 13: Olivier Faugeras, Three-dimensional computer vision: a geometric viewpoint, MIT Press, Cambridge, MA, 1993
• 14: Steven J. Gortler , Radek Grzeszczuk , Richard Szeliski , Michael F. Cohen, The lumigraph, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.43-54, August 1996 [doi>10.1145/237170.237200]
• 15: F.R. Hampel, P.J. Rousseeuw, E.M. Ronchetti, and W.A. Stahel. Robust Statistics. John Wiley & Sons, New York, 1986.
• 16: C. Harris and M.J. Stephens. A combined corner and edge detector. In Alvey Vision Conference, pages 147-152, 1988.
• 17: David J. Heeger , James R. Bergen, Pyramid-based texture analysis/synthesis, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.229-238, September 1995 [doi>10.1145/218380.218446]
• 18: Wolfgang Heidrich , Katja Daubert , Jan Kautz , Hans-Peter Seidel, Illuminating micro geometry based on precomputed visibility, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.455-464, July 2000 [doi>10.1145/344779.344984]
• 19: Berthold K. P. Horn , Michael J. Brooks, The variational approach to shape from shading, Computer Vision, Graphics, and Image Processing, v.33 n.2, p.174-208, Feb. 1986 [doi>10.1016/0734-189X(86)90114-3]
• 20: J.J. Koenderink and A.J. van Doorn. Illuminance texture due to surface mesostructure. J. Opt. Soc. Am.A, 13(3):452-463, 1996.
• 21: Y.G. Leclerc and A.F. Bobick. The direct computation of height from shading. In Proc. of IEEE Conf. on Comp. Vision and Patt. Recog., pages 552-558, 1991.
• 22: Thomas Leung , Jitendra Malik, Recognizing Surfaces Using Three-Dimensional Textons, Proceedings of the International Conference on Computer Vision-Volume 2, p.1010, September 20-25, 1999
• 23: Marc Levoy , Pat Hanrahan, Light field rendering, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.31-42, August 1996 [doi>10.1145/237170.237199]
• 24: S. R. Marschner, S. H. Westin, E. P. F. Lafortune, K. E. Torrance, and D. P. Greenberg. Image-based brdf measurement including human skin. In 10th Eurographics Workshop on Rendering, pages 139-152, 1999.
• 25: Leonard McMillan , Gary Bishop, Plenoptic modeling: an image-based rendering system, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.39-46, September 1995 [doi>10.1145/218380.218398]
• 26: D.M. Mount. ANN Programming Manual. Dept. Comput. Sci., Univ. of Maryland, College Park, Maryland, 1998.
• 27: S.K. Nayar, K. Ikeuchi, and T. Kanade. Determining shape and reflectance of hybrid surfaces by photometric sampling. IEEE Trans. Robotics and Automa-tion, 6(4):418-431, 1990.
• 28: K. Nishino, Y. Sato, and K. Ikeuchi. Eigen-texture method: appearance compression based on 3d model. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR'99), pages 618-624, 1999.
• 29: Emil Praun , Adam Finkelstein , Hugues Hoppe, Lapped textures, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.465-470, July 2000 [doi>10.1145/344779.344987]
• 30: William H. Press , Brian P. Flannery , Saul A. Teukolsky , William T. Vetterling, Numerical recipes in C: the art of scientific computing, Cambridge University Press, New York, NY, 1988
• 31: Holly E. Rushmeier , Gabriel Taubin , André Guéziec, Appying Shape from Lighting Variation to Bump Map Capture, Proceedings of the Eurographics Workshop on Rendering Techniques '97, p.35-44, June 16-18, 1997
• 32: Yoichi Sato , Mark D. Wheeler , Katsushi Ikeuchi, Object shape and reflectance modeling from observation, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.379-387, August 1997 [doi>10.1145/258734.258885]
• 33: Steven M. Seitz , Charles R. Dyer, Photorealistic Scene Reconstruction by Voxel Coloring, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.1067, June 17-19, 1997
• 34: Jonathan Shade , Steven Gortler , Li-wei He , Richard Szeliski, Layered depth images, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.231-242, July 1998 [doi>10.1145/280814.280882]
• 35: Heung-Yeung Shum , Li-Wei He, Rendering with concentric mosaics, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.299-306, July 1999 [doi>10.1145/311535.311573]
• 36: E. Simoncelli and J. Portilla. Texture characterization via joint statistics of wavelet coefficient magnitudes. In Fifth International Conference on Image Processing, Vol.1, pages 62-66, 1998.
• 37: Richard Szeliski , Heung-Yeung Shum, Creating full view panoramic image mosaics and environment maps, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.251-258, August 1997 [doi>10.1145/258734.258861]
• 38: Li-Yi Wei , Marc Levoy, Fast texture synthesis using tree-structured vector quantization, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.479-488, July 2000 [doi>10.1145/344779.345009]
• 39: Daniel N. Wood , Daniel I. Azuma , Ken Aldinger , Brian Curless , Tom Duchamp , David H. Salesin , Werner Stuetzle, Surface light fields for 3D photography, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.287-296, July 2000 [doi>10.1145/344779.344925]
• 40: Robert J. Woodham, Photometric method for determining surface orientation from multiple images, Shape from shading, MIT Press, Cambridge, MA, 1989
• 41: Y. Xu, B. Guo, and H.-Y. Shum. Chaos mosaic: Fast and memory efficient texture synthesis. Technical Report MSR-TR-2000-32, Microsoft Research, 2000.
• 42: Yizhou Yu , Jitendra Malik, Modeling and editing real scenes with image-based techniques, 2000
• 43: Yizhou Yu , Paul Debevec , Jitendra Malik , Tim Hawkins, Inverse global illumination: recovering reflectance models of real scenes from photographs, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.215-224, July 1999 [doi>10.1145/311535.311559]
• 44: Yizhou Yu , Jitendra Malik, Recovering photometric properties of architectural scenes from photographs, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.207-217, July 1998 [doi>10.1145/280814.280874]
• 45: Song Chun Zhu , Yingnian Wu , David Mumford, Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling, International Journal of Computer Vision, v.27 n.2, p.107-126, Mar 1998 [doi>10.1023/A:1007925832420]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Modeling and recovery of physical attributes

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Color, shading, shadowing, and texture
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Color; Photometry; Shading

General Terms:
Algorithms, Design, Performance, Reliability, Theory

Keywords:
bidirectional texture functions, image-based rendering, photometric stereo, reflectance and shading models, shape-from-shading, texture synthesis