Jump map-based interactive texture synthesis

ABSTRACT

We present techniques for accelerated texture synthesis from example images. The key idea of our approach is to divide the task into two phases: analysis, and synthesis. During the analysis phase, which is performed once per sample texture, we generate a <i>jump map</i>. Using the jump map, the synthesis phase is capable of synthesizing texture similar to the analyzed example at interactive rates. We describe two such synthesis phase algorithms: one for creating images, and one for directly texturing manifold surfaces. We produce texture images at rates comparable to the fastest alternative algorithms, and produce textured surfaces an order of magnitude faster than current alternative approaches. We further develop a new, faster patch-based algorithm for image synthesis, which improves the quality of our results on ordered textures. We show how controls used for specifying texture synthesis on surfaces may be used on images as well, allowing interesting new image-based effects, and highlight modelling applications enabled by the speed of our approach.

REFERENCES

• 1: Okan Arikan , D. A. Forsyth, Interactive motion generation from examples, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 2: Michael Ashikhmin, Synthesizing natural textures, Proceedings of the 2001 symposium on Interactive 3D graphics, p.217-226, March 2001 [doi>10.1145/364338.364405]
• 3: Michael Ashikhmin, Fast Texture Transfer, IEEE Computer Graphics and Applications, v.23 n.4, p.38-43, July 2003 [doi>10.1109/MCG.2003.1210863]
• 4: Bagomjakov, A. and Gotsman, C. 2002. Universal rendering sequences for transparent vertex caching of progressive meshes. Comput. Graph. Forum 21, 2, 137--148.
• 5: Bertalmio, M., Vese, L., Sapiro, G., and Osher, S. 2003. Simultaneous structure and texture image inpainting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2003). Vol. 2. IEEE Computer Society, 707--712.
• 6: Nathan A. Carr , John C. Hart, Meshed atlases for real-time procedural solid texturing, ACM Transactions on Graphics (TOG), v.21 n.2, p.106-131, April 2002 [doi>10.1145/508357.508360]
• 7: Michael F. Cohen , Jonathan Shade , Stefan Hiller , Oliver Deussen, Wang Tiles for image and texture generation, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 8: Criminisi, A., Perez, P., and Toyama, K. 2003. Object removal by exemplar-based inpainting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2003). Vol. 2. IEEE Computer Society, 721--728.
• 9: Jeremy S. De Bonet, Multiresolution sampling procedure for analysis and synthesis of texture images, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.361-368, August 1997 [doi>10.1145/258734.258882]
• 10: Dischler, J.-M., Maritaud, K., Lévy, B., and Ghazanfarpour, D. 2002. Texture particles. In Eurographics 2002 Conference Proceedings. Eurographics Association, Saarbr ucken, Germany.
• 11: Iddo Drori , Daniel Cohen-Or , Hezy Yeshurun, Fragment-based image completion, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 12: David S. Ebert , F. Kenton Musgrave , Darwyn Peachey , Ken Perlin , Steven Worley, Texturing and modeling: a procedural approach, Academic Press Professional, Inc., San Diego, CA, 1994
• 13: Alexei A. Efros , William T. Freeman, Image quilting for texture synthesis and transfer, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.341-346, August 2001 [doi>10.1145/383259.383296]
• 14: Alexei A. Efros , Thomas K. Leung, Texture Synthesis by Non-Parametric Sampling, Proceedings of the International Conference on Computer Vision-Volume 2, p.1033, September 20-25, 1999
• 15: Kurt W. Fleischer , David H. Laidlaw , Bena L. Currin , Alan H. Barr, Cellular texture generation, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.239-248, September 1995 [doi>10.1145/218380.218447]
• 16: Harrison, P. 2001. A non-hierarchical procedure for re-synthesis of complex textures. In Winter School of Computer Graphics (WSCG'01) 2001, V. Skala, Ed. 190--197.
• 17: David J. Heeger , James R. Bergen, Pyramid-based texture analysis/synthesis, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.229-238, September 1995 [doi>10.1145/218380.218446]
• 18: Aaron Hertzmann , Charles E. Jacobs , Nuria Oliver , Brian Curless , David H. Salesin, Image analogies, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.327-340, August 2001 [doi>10.1145/383259.383295]
• 19: Hertzmann, A. and Seitz, S. M. 2003. Shape and materials by example: A photometric stereo approach. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR '03). Vol. 1. IEEE Computer Society, Madison, WI, 533--540.
• 20: Jia, J. and Tang, C. K. 2003. Image repairing: Robust image synthesis by adaptive nd tensor voting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2003). Vol. 1. IEEE Computer Society, 643--650.
• 21: Jollife, I. T. 1986. Principal Component Analysis. Springer-Verlag, New York.
• 22: Lucas Kovar , Michael Gleicher , Frédéric Pighin, Motion graphs, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 23: Vivek Kwatra , Arno Schödl , Irfan Essa , Greg Turk , Aaron Bobick, Graphcut textures: image and video synthesis using graph cuts, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 24: Jehee Lee , Jinxiang Chai , Paul S. A. Reitsma , Jessica K. Hodgins , Nancy S. Pollard, Interactive control of avatars animated with human motion data, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 25: Justin Legakis , Julie Dorsey , Steven Gortler, Feature-based cellular texturing for architectural models, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.309-316, August 2001 [doi>10.1145/383259.383293]
• 26: Thomas Leung , Jitendra Malik, Recognizing Surfaces Using Three-Dimensional Textons, Proceedings of the International Conference on Computer Vision-Volume 2, p.1010, September 20-25, 1999
• 27: Lin Liang , Ce Liu , Ying-Qing Xu , Baining Guo , Heung-Yeung Shum, Real-time texture synthesis by patch-based sampling, ACM Transactions on Graphics (TOG), v.20 n.3, p.127-150, July 2001 [doi>10.1145/501786.501787]
• 28: Liu, C., Shum, H.-Y., and Zhang, C.-S. 2001. A two-step approach to hallucinating faces: Global parametric model and local non-parametric model. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR '01). Vol. 1. IEEE Computer Society, Kauai, Hawaii, 192--198.
• 29: Sebastian Magda , David Kriegman, Fast texture synthesis on arbitrary meshes, Proceedings of the 14th Eurographics workshop on Rendering, June 25-27, 2003, Leuven, Belgium
• 30: Mount, D. M. 1998. Ann programming manual. Tech. rep., Department of Computer Science, University of Maryland, College Park, Maryland.
• 31: Andrew Nealen , Marc Alexa, Hybrid texture synthesis, Proceedings of the 14th Eurographics workshop on Rendering, June 25-27, 2003, Leuven, Belgium
• 32: Fabrice Neyret , Marie-Paule Cani, Pattern-based texturing revisited, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.235-242, July 1999 [doi>10.1145/311535.311561]
• 33: Dan Piponi , George Borshukov, Seamless texture mapping of subdivision surfaces by model pelting and texture blending, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.471-478, July 2000 [doi>10.1145/344779.344990]
• 34: Emil Praun , Adam Finkelstein , Hugues Hoppe, Lapped textures, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.465-470, July 2000 [doi>10.1145/344779.344987]
• 35: Sagan, H. 1994. Space-Filling Curves. Springer-Verlag, New York.
• 36: Pedro V. Sander , Steven J. Gortler , John Snyder , Hugues Hoppe, Signal-specialized parametrization, Proceedings of the 13th Eurographics workshop on Rendering, June 26-28, 2002, Pisa, Italy
• 37: Arno Schödl , Richard Szeliski , David H. Salesin , Irfan Essa, Video textures, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.489-498, July 2000 [doi>10.1145/344779.345012]
• 38: Alla Sheffer , John C. Hart, Seamster: inconspicuous low-distortion texture seam layout, Proceedings of the conference on Visualization '02, October 27-November 01, 2002, Boston, Massachusetts
• 39: Cyril Soler , Marie-Paule Cani , Alexis Angelidis, Hierarchical pattern mapping, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 40: Xin Tong , Jingdan Zhang , Ligang Liu , Xi Wang , Baining Guo , Heung-Yeung Shum, Synthesis of bidirectional texture functions on arbitrary surfaces, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 41: Greg Turk, Texture synthesis on surfaces, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.347-354, August 2001 [doi>10.1145/383259.383297]
• 42: Li-Yi Wei , Marc Levoy, Fast texture synthesis using tree-structured vector quantization, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.479-488, July 2000 [doi>10.1145/344779.345009]
• 43: Li-Yi Wei , Marc Levoy, Texture synthesis over arbitrary manifold surfaces, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.355-360, August 2001 [doi>10.1145/383259.383298]
• 44: Wei, L.-Y. and Levoy, M. 2002. Order-independent texture synthesis. Tech. Rep. TR-2002-01, Computer Science Department, Stanford University. Apr.
• 45: Tomihisa Welsh , Michael Ashikhmin , Klaus Mueller, Transferring color to greyscale images, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 46: Xu, Y.-Q., Guo, B., and Shum, H. 2000. Chaos mosaic: Fast and memory efficient texture synthesis. Tech. Rep. MSR-TR-2000-32, Microsoft Research. April.
• 47: Lexing Ying , Aaron Hertzmann , Henning Biermann , Denis Zorin, Texture and Shape Synthesis on Surfaces, Proceedings of the 12th Eurographics Workshop on Rendering Techniques, p.301-312, June 25-27, 2001
• 48: Steve Zelinka , Michael Garland, Towards real-time texture synthesis with the jump map, Proceedings of the 13th Eurographics workshop on Rendering, June 26-28, 2002, Pisa, Italy
• 49: Steve Zelinka , Michael Garland, Interactive texture synthesis on surfaces using jump maps, Proceedings of the 14th Eurographics workshop on Rendering, June 25-27, 2003, Leuven, Belgium
• 50: Jingdan Zhang , Kun Zhou , Luiz Velho , Baining Guo , Heung-Yeung Shum, Synthesis of progressively-variant textures on arbitrary surfaces, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Color, shading, shadowing, and texture

General Terms:
Algorithms

Keywords:
Interactive texture synthesis, jump maps, texturing surfaces