Animating Chinese paintings through stroke-based decomposition

ABSTRACT

This article proposes a technique to animate a Chinese style painting given its image. We first extract descriptions of the brush strokes that hypothetically produced it. The key to the extraction process is the use of a brush stroke library, which is obtained by digitizing single brush strokes drawn by an experienced artist. The steps in our extraction technique are first to segment the input image, then to find the best set of brush strokes that fit the regions, and, finally, to refine these strokes to account for local appearance. We model a single brush stroke using its skeleton and contour, and we characterize texture variation within each stroke by sampling perpendicularly along its skeleton. Once these brush descriptions have been obtained, the painting can be animated at the brush stroke level. In this article, we focus on Chinese paintings with relatively sparse strokes. The animation is produced using a graphical application we developed. We present several animations of real paintings using our technique.

REFERENCES

• 1: Marc Alexa , Daniel Cohen-Or , David Levin, As-rigid-as-possible shape interpolation, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.157-164, July 2000 [doi>10.1145/344779.344859]
• 2: William A. Barrett , Alan S. Cheney, Object-based image editing, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 3: Bill Baxter , Vincent Scheib , Ming C. Lin , Dinesh Manocha, DAB: interactive haptic painting with 3D virtual brushes, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.461-468, August 2001 [doi>10.1145/383259.383313]
• 4: Dorothea Blostein , Lippold Haken, Using Diagram Generation Software to Improve Diagram Recognition: A Case Study of Music Notation, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.21 n.11, p.1121-1136, November 1999 [doi>10.1109/34.809106]
• 5: Chen, H. Ed. 1994. Encyclopaedia of China, Film Volume (in Chinese). Encyclopaedia of China Press.
• 6: Chen, J. and Zhang, J. Eds. 1995. Dictionary of Chinese Films (in Chinese). Shanghai Dictionary Press.
• 7: Chuang, Y.-Y. et al. 2001. A Bayesian approach to digital matting. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'01). Kauai, Hawaii, vol. II, 264--271.
• 8: Dorin Comaniciu , Peter Meer, Mean Shift: A Robust Approach Toward Feature Space Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.5, p.603-619, May 2002 [doi>10.1109/34.1000236]
• 9: Eric Daniels, Deep canvas in Disney's Tarzan, ACM SIGGRAPH 99 Conference abstracts and applications, p.200, August 08-13, 1999, Los Angeles, California, United States [doi>10.1145/311625.312010]
• 10: Doug DeCarlo , Anthony Santella, Stylization and abstraction of photographs, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 11: Farid, H. and Adelson, E. 1999. Separating reflections from images by use of independent components analysis. J. Optical Society America 16, 9, 2136--2145.
• 12: Feldman, J. A. and Yakimovsky, Y. 1974. Decision theory and artificial intelligence: I. a semantics-based region analyzer. Artific. Intell. 5, 4, 349--371.
• 13: David A. Forsyth , Jean Ponce, Computer Vision: A Modern Approach, Prentice Hall Professional Technical Reference, 2002
• 14: Bruce Gooch , Greg Coombe , Peter Shirley, Artistic Vision: painterly rendering using computer vision techniques, Proceedings of the 2nd international symposium on Non-photorealistic animation and rendering, June 03-05, 2002, Annecy, France [doi>10.1145/508530.508545]
• 15: Hertzmann, A. 1999. Introduction to 3D non-photorealistic rendering: Silhouettes and outlines. S. Green, Ed. Non-Photorealistic Rendering. SIGGRAPH 99 Course Notes.
• 16: Youichi Horry , Ken-Ichi Anjyo , Kiyoshi Arai, Tour into the picture: using a spidery mesh interface to make animation from a single image, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.225-232, August 1997 [doi>10.1145/258734.258854]
• 17: Hsu, S., Lee, I., Lou, C., and Siu, S. 1999. Software. Creature House (www.creaturehouse.com).
• 18: Siu Chi Hsu , Irene H. H. Lee, Drawing and animation using skeletal strokes, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.109-118, July 1994 [doi>10.1145/192161.192186]
• 19: Anil K. Jain, Fundamentals of digital image processing, Prentice-Hall, Inc., Upper Saddle River, NJ, 1989
• 20: S. H. Joseph , T. P. Pridmore, Knowledge-Directed Interpretation of Mechanical Engineering Drawings, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.14 n.9, p.928-940, September 1992 [doi>10.1109/34.161351]
• 21: Judd, D. B. and Wyszecki, G. 1975. Color in Business, Science, and Industry. John Wiley and Sons, New York, NY.
• 22: Robert D. Kalnins , Lee Markosian , Barbara J. Meier , Michael A. Kowalski , Joseph C. Lee , Philip L. Davidson , Matthew Webb , John F. Hughes , Adam Finkelstein, WYSIWYG NPR: drawing strokes directly on 3D models, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 23: Kumar, K. and Desai, U. 1999. Joint segmentation and image interpretation. Patt. Recog. 32, 4, 577--589.
• 24: Levin, A. and Weiss, Y. 2004. User assisted separation of reflections from a single image using a sparsity prior. In European Conference on Computer Vision (ECCV'04). 602--613.
• 25: Levin, A., Zomet, A., and Weiss, Y. 2004. Separating reflections from a single image using local features. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'04). 306--313.
• 26: Peter Litwinowicz , Lance Williams, Animating images with drawings, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.409-412, July 1994 [doi>10.1145/192161.192270]
• 27: Liu, L. and Sclaroff, S. 2001. Region segmentation via deformable model-guided split and merge. In Proceedings of the International Conference on Computer Vision (ICCV'01), vol. 1. 98--104.
• 28: Jose L. Marroquin , Edgar Arce Santana , Salvador Botello, Hidden Markov Measure Field Models for Image Segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.11, p.1380-1387, November 2003 [doi>10.1109/TPAMI.2003.1240112]
• 29: Jan A. Mulder , Alan K. Mackworth , Willian S. Havens, Knowledge Structuring and Constraint Satisfaction: The Mapsee Approach, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.10 n.6, p.866-879, November 1988 [doi>10.1109/34.9108]
• 30: Anke Neumann, Graphical Gaussian Shape Models and Their Application to Image Segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.3, p.316-329, March 2003 [doi>10.1109/TPAMI.2003.1182095]
• 31: Thomas Porter , Tom Duff, Compositing digital images, Proceedings of the 11th annual conference on Computer graphics and interactive techniques, p.253-259, January 1984
• 32: Michael P. Salisbury , Sean E. Anderson , Ronen Barzel , David H. Salesin, Interactive pen-and-ink illustration, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.101-108, July 1994 [doi>10.1145/192161.192185]
• 33: Philip J. Schneider, An algorithm for automatically fitting digitized curves, Graphics gems, Academic Press Professional, Inc., San Diego, CA, 1990
• 34: Stan Sclaroff , Lifeng Liu, Deformable Shape Detection and Description via Model-Based Region Grouping, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.5, p.475-489, May 2001 [doi>10.1109/34.922706]
• 35: Thomas W. Sederberg , Rida T. Farouki, Approximation by Interval Bezier Curves, IEEE Computer Graphics and Applications, v.12 n.5, p.87-95, September 1992 [doi>10.1109/38.156018]
• 36: Smith, R. and Lloyd, E. J. 1997. Art School. Dorling Kindersley Ltd, Inc.
• 37: Steve Strassmann, Hairy brushes, Proceedings of the 13th annual conference on Computer graphics and interactive techniques, p.225-232, August 1986
• 38: Su, S., Xu, Y., Shum, H., and Chen, F. 2002. Simulating artistic brushstrokes using interval splines. In Proceedings of the 5th IASTED International Conference on Computer Graphics and Imaging. Kauai, Hawaii, 85--90.
• 39: Szeliski, R., Avidan, S., and Anandan, P. 2000. Layer extraction from multiple images containing reflections and transparency. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'00). Hilton Head, SC. vol. I, 246--253.
• 40: Tenenbaum, J. M. and Barrow, H. G. 1977. Experiments in interpretation-guided segmentation. Artific. Intell. 8, 3, 241--274.
• 41: Veltkamp, R. C. and Hagedoorn, M. 1999. State of the art in shape matching. Tech. rep. UU-CS-1999-27, Utrecht University, Netherlands.
• 42: Wang, C. and Srihari, S. 1988. A framework for object recognition in a visually complex environment and its application to locating address blocks on mail pieces. Intl. J. Comput. Vision 2, 125--151.
• 43: Jin Wang , Jack Jean, Segmentation of merged characters by neural networks and shortest-path, Proceedings of the 1993 ACM/SIGAPP symposium on Applied computing: states of the art and practice, p.762-769, February 14-16, 1993, Indianapolis, Indiana, United States [doi>10.1145/162754.168698]
• 44: Wang, S. and Siskind, J. M. 2003. Image segmentation with ratio cut. IEEE Trans. Pattern Anal. Mach. Intell. 25, 6, 675--690.
• 45: Songhua Xu , Min Tang , Francis C. M. Lau , Yunhe Pan, Virtual hairy brush for painterly rendering, Graphical Models, v.66 n.5, p.263-302, September 2004 [doi>10.1016/j.gmod.2004.05.006]
• 46: Richard Zanibbi , Dorothea Blostein , James R. Cordy, Recognizing Mathematical Expressions Using Tree Transformation, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.11, p.1455-1467, November 2002 [doi>10.1109/TPAMI.2002.1046157]
• 47: R. W. Zhou , C. Quek , G. S. Ng, A novel single-pass thinning algorithm and an effective set of performance criteria, Pattern Recognition Letters, v.16 n.12, p.1267-1275, Dec. 1995 [doi>10.1016/0167-8655(95)00078-X]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.3 Picture/Image Generation

General Terms:
Algorithms, Design

Keywords:
Computer animation, image editing, image segmentation, image-based modeling and rendering, non-photorealistic rendering