Inverse texture synthesis

ABSTRACT

The quality and speed of most texture synthesis algorithms depend on a 2D input sample that is small and contains enough texture variations. However, little research exists on how to acquire such sample. For homogeneous patterns this can be achieved via manual cropping, but no adequate solution exists for inhomogeneous or globally varying textures, i.e. patterns that are local but not stationary, such as rusting over an iron statue with appearance conditioned on varying moisture levels.

We present inverse texture synthesis to address this issue. Our inverse synthesis runs in the opposite direction with respect to traditional forward synthesis: given a large globally varying texture, our algorithm automatically produces a small texture compaction that best summarizes the original. This small compaction can be used to reconstruct the original texture or to re-synthesize new textures under user-supplied controls. More important, our technique allows real-time synthesis of globally varying textures on a GPU, where the texture memory is usually too small for large textures. We propose an optimization framework for inverse texture synthesis, ensuring that each input region is properly encoded in the output compaction. Our optimization process also automatically computes orientation fields for anisotropic textures containing both low- and high-frequency regions, a situation difficult to handle via existing techniques.

REFERENCES

• 1: Michael Ashikhmin, Synthesizing natural textures, Proceedings of the 2001 symposium on Interactive 3D graphics, p.217-226, March 2001 [doi>10.1145/364338.364405]
• 2: Shai Avidan , Ariel Shamir, Seam carving for content-aware image resizing, ACM SIGGRAPH 2007 papers, August 05-09, 2007, San Diego, California
• 3: Adam W. Bargteil , Funshing Sin , Jonathan E. Michaels , Tolga G. Goktekin , James F. O'Brien, A texture synthesis method for liquid animations, Proceedings of the 2006 ACM SIGGRAPH/Eurographics symposium on Computer animation, September 02-04, 2006, Vienna, Austria
• 4: Stephen Brooks , Neil Dodgson, Self-similarity based texture editing, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 5: Brian Cabral , Leith Casey Leedom, Imaging vector fields using line integral convolution, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.263-270, August 02-06, 0093, Anaheim, CA [doi>10.1145/166117.166151]
• 6: Michael F. Cohen , Jonathan Shade , Stefan Hiller , Oliver Deussen, Wang Tiles for image and texture generation, ACM SIGGRAPH 2003 Papers, July 27-31, 2003, San Diego, California
• 7: Iddo Drori , Daniel Cohen-Or , Hezy Yeshurun, Fragment-based image completion, ACM SIGGRAPH 2003 Papers, July 27-31, 2003, San Diego, California
• 8: Alexei A. Efros , William T. Freeman, Image quilting for texture synthesis and transfer, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.341-346, August 2001 [doi>10.1145/383259.383296]
• 9: Alexei A. Efros , Thomas K. Leung, Texture Synthesis by Non-Parametric Sampling, Proceedings of the International Conference on Computer Vision-Volume 2, p.1033, September 20-25, 1999
• 10: Hui Fang , John C. Hart, Textureshop: texture synthesis as a photograph editing tool, ACM SIGGRAPH 2004 Papers, August 08-12, 2004, Los Angeles, California
• 11: Allen Gersho , Robert M. Gray, Vector quantization and signal compression, Kluwer Academic Publishers, Norwell, MA, 1991
• 12: Jinwei Gu , Chien-I Tu , Ravi Ramamoorthi , Peter Belhumeur , Wojciech Matusik , Shree Nayar, Time-varying surface appearance: acquisition, modeling and rendering, ACM SIGGRAPH 2006 Papers, July 30-August 03, 2006, Boston, Massachusetts
• 13: Jianwei Han , Kun Zhou , Li-Yi Wei , Minmin Gong , Hujun Bao , Xinming Zhang , Baining Guo, Fast example-based surface texture synthesis via discrete optimization, The Visual Computer: International Journal of Computer Graphics, v.22 n.9, p.918-925, September 2006 [doi>10.1007/s00371-006-0078-3]
• 14: Pat Hanrahan , Paul Haeberli, Direct WYSIWYG painting and texturing on 3D shapes, Proceedings of the 17th annual conference on Computer graphics and interactive techniques, p.215-223, September 1990, Dallas, TX, USA
• 15: David J. Heeger , James R. Bergen, Pyramid-based texture analysis/synthesis, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.229-238, September 1995 [doi>10.1145/218380.218446]
• 16: Aaron Hertzmann , Charles E. Jacobs , Nuria Oliver , Brian Curless , David H. Salesin, Image analogies, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.327-340, August 2001 [doi>10.1145/383259.383295]
• 17: Nebojsa Jojic , Brendan J. Frey , Anitha Kannan, Epitomic analysis of appearance and shape, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.34, October 13-16, 2003
• 18: Kannan, A., Winn, J., and Rother, C. 2007. Clustering appearance and shape by learning jigsaws. In Advances in Neural Information Processing Systems 19.
• 19: Johannes Kopf , Chi-Wing Fu , Daniel Cohen-Or , Oliver Deussen , Dani Lischinski , Tien-Tsin Wong, Solid texture synthesis from 2D exemplars, ACM SIGGRAPH 2007 papers, August 05-09, 2007, San Diego, California
• 20: Vivek Kwatra , Irfan Essa , Aaron Bobick , Nipun Kwatra, Texture optimization for example-based synthesis, ACM SIGGRAPH 2005 Papers, July 31-August 04, 2005, Los Angeles, California
• 21: Kwatra, V., Adalsteinsson, D., Kim, T., Kwatra, N., Carlson, M., and Lin, M. 2007. Texturing fluids. IEEE Trans. Visualization and Computer Graphics 13, 5, 939--952.
• 22: Sylvain Lefebvre , Hugues Hoppe, Parallel controllable texture synthesis, ACM SIGGRAPH 2005 Papers, July 31-August 04, 2005, Los Angeles, California
• 23: Sylvain Lefebvre , Hugues Hoppe, Appearance-space texture synthesis, ACM SIGGRAPH 2006 Papers, July 30-August 03, 2006, Boston, Massachusetts
• 24: Thomas Leung , Jitendra Malik, Representing and Recognizing the Visual Appearance of Materials using Three-dimensional Textons, International Journal of Computer Vision, v.43 n.1, p.29-44, June 2001 [doi>10.1023/A:1011126920638]
• 25: Yanxi Liu , Wen-Chieh Lin , James Hays, Near-regular texture analysis and manipulation, ACM SIGGRAPH 2004 Papers, August 08-12, 2004, Los Angeles, California
• 26: Jianye Lu , Athinodoros S. Georghiades , Andreas Glaser , Hongzhi Wu , Li-Yi Wei , Baining Guo , Julie Dorsey , Holly Rushmeier, Context-aware textures, ACM Transactions on Graphics (TOG), v.26 n.1, p.3-es, January 2007 [doi>10.1145/1189762.1189765]
• 27: Wojciech Matusik , Matthias Zwicker , Frédo Durand, Texture design using a simplicial complex of morphable textures, ACM SIGGRAPH 2005 Papers, July 31-August 04, 2005, Los Angeles, California
• 28: Sylvain Paris , Hector M. Briceño , François X. Sillion, Capture of hair geometry from multiple images, ACM SIGGRAPH 2004 Papers, August 08-12, 2004, Los Angeles, California
• 29: Perona, P., and Malik, J. 1990. Detecting and localizing edges composed of steps, peaks and roofs. In ICCV '90, 52--57.
• 30: Popat, K., and Picard, R. W. 1997. Cluster based probability model and its application to image and texture processing. IEEE Trans. Image Processing 6, 2, 268--284.
• 31: Javier Portilla , Eero P. Simoncelli, A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients, International Journal of Computer Vision, v.40 n.1, p.49-70, Oct. 2000 [doi>10.1023/A:1026553619983]
• 32: Xuejie Qin , Yee-Hong Yang, Basic Gray Level Aura Matrices: Theory and its Application to Texture Synthesis, Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, p.128-135, October 17-20, 2005 [doi>10.1109/ICCV.2005.43]
• 33: Ritter, L., Li, W., Curless, B., Agrawala, M., and Salesin, D. 2006. Painting with texture. In Eurographics Symposium on Rendering, 371--376.
• 34: Xin Tong , Jingdan Zhang , Ligang Liu , Xi Wang , Baining Guo , Heung-Yeung Shum, Synthesis of bidirectional texture functions on arbitrary surfaces, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 35: Greg Turk, Texture synthesis on surfaces, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.347-354, August 2001 [doi>10.1145/383259.383297]
• 36: Jiaping Wang , Xin Tong , Stephen Lin , Minghao Pan , Chao Wang , Hujun Bao , Baining Guo , Heung-Yeung Shum, Appearance manifolds for modeling time-variant appearance of materials, ACM SIGGRAPH 2006 Papers, July 30-August 03, 2006, Boston, Massachusetts
• 37: Huamin Wang , Yonatan Wexler , Eyal Ofek , Hugues Hoppe, Factoring repeated content within and among images, ACM SIGGRAPH 2008 papers, August 11-15, 2008, Los Angeles, California
• 38: Li-Yi Wei , Marc Levoy, Fast texture synthesis using tree-structured vector quantization, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.479-488, July 2000 [doi>10.1145/344779.345009]
• 39: Li-Yi Wei , Marc Levoy, Texture synthesis over arbitrary manifold surfaces, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.355-360, August 2001 [doi>10.1145/383259.383298]
• 40: Jingdan Zhang , Kun Zhou , Luiz Velho , Baining Guo , Heung-Yeung Shum, Synthesis of progressively-variant textures on arbitrary surfaces, ACM SIGGRAPH 2003 Papers, July 27-31, 2003, San Diego, California
• 41: Ziou, D., and Tabbone, S. 1998. Edge detection techniques - an overview. International Journal of Pattern Recognition and Image Analysis 8, 537--559.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Color, shading, shadowing, and texture

Additional Classification:
  E. Data
  E.4 CODING AND INFORMATION THEORY
      Subjects: Data compaction and compression

  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.6 Segmentation
          Subjects: Edge and feature detection
      I.4.7 Feature Measurement
          Subjects: Texture

General Terms:
Algorithms, Design, Measurement

Keywords:
GPU techniques, texture mapping, texture synthesis