PicToon: a personalized image-based cartoon system

ABSTRACT

In this paper, we present PicToon, a cartoon system which can generate a personalized cartoon face from an input Picture. PicToon is easy to use and requires little user interaction. Our system consists of three major components: an image-based Cartoon Generator, an interactive Cartoon Editor for exaggeration, and a speech-driven Cartoon Animator. First, to capture an artistic style, the cartoon generation is decoupled into two processes: sketch generation and stroke rendering. An example-based approach is taken to automatically generate sketch lines which depict the facial structure. An inhomogeneous non-parametric sampling plus a flexible facial template is employed to extract the vector-based facial sketch. Various styles of strokes can then be applied. Second, with the pre-designed templates in Cartoon Editor, the user can easily make the cartoon exaggerated or more expressive. Third, a real-time lip-syncing algorithm is also developed that recovers a statistical audio-visual mapping between the character's voice and the corresponding lip configuration. Experimental results demonstrate the effectiveness of our system.

REFERENCES

• 1: C. Bouman. CLUSTER: An unsupervised algorithm for modeling gaussian mixtures. http://dynamo.ecn.purdue.edu/ ~bouman/.
• 2: Matthew Brand, Voice puppetry, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.21-28, July 1999 [doi>10.1145/311535.311537]
• 3: Christoph Bregler , Michele Covell , Malcolm Slaney, Video Rewrite: driving visual speech with audio, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.353-360, August 1997 [doi>10.1145/258734.258880]
• 4: T. Cootes and C. Taylor. Statistical models of appearance for computer version. Technical report, University of Manchester, Manchester M13 9PT, U.K., 2000.
• 5: T. Cootes and C. Taylor. Constrained active appearance models. In ICCV01, pages I: 748--754, 2001.
• 6: Frédo Durand , Victor Ostromoukhov , Mathieu Miller , Francois Duranleau , Julie Dorsey, Decoupling Strokes and High-Level Attributes for Interactive Traditional Drawing, Proceedings of the 12th Eurographics Workshop on Rendering Techniques, p.71-82, June 25-27, 2001
• 7: B. Edwards. The new drawing on the right side of the brain. J P Tarcher, 1999.
• 8: Alexei A. Efros , Thomas K. Leung, Texture Synthesis by Non-Parametric Sampling, Proceedings of the International Conference on Computer Vision-Volume 2, p.1033, September 20-25, 1999
• 9: H. Chen, N. Zheng, L. Liang, Y. Li, Y. Xu, and H. Shum. A personalized cartoon system (in chinese). In Chinagraph'02, Beijing, China, 2002.
• 10: Siu Chi Hsu , Irene H. H. Lee, Drawing and animation using skeletal strokes, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.109-118, July 1994 [doi>10.1145/192161.192186]
• 11: http://www.macromedia.com/software/.
• 12: K. Kumagai. How to draw expressive face. Tianjin People's Fine Arts Publishing House, China, 2000.
• 13: Peter C. Litwinowicz, Inkwell: A 2-D animation system, ACM SIGGRAPH Computer Graphics, v.25 n.4, p.113-122, July 1991
• 14: S. Morishima, K. Aizawa, and H. Harashima. An intelligent facial image coding driven by speech and phoneme. In IEEE ICASSP, 1989.
• 15: H. Munce, editor. How to draw the human figure. Cortinal Learning International, Inc., 1983.
• 16: J. D. Northrup , Lee Markosian, Artistic silhouettes: a hybrid approach, Proceedings of the 1st international symposium on Non-photorealistic animation and rendering, p.31-37, June 05-07, 2000, Annecy, France [doi>10.1145/340916.340920]
• 17: Victor Ostromoukhov, Digital facial engraving, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.417-424, July 1999 [doi>10.1145/311535.311604]
• 18: Lawrence Rabiner , Biing-Hwang Juang, Fundamentals of speech recognition, Prentice-Hall, Inc., Upper Saddle River, NJ, 1993
• 19: Frank Van Reeth, Integrating 2 1/2-D Computer Animation Techniques for Supporting Traditional Animation, Proceedings of the Computer Animation, p.118, June 03-04, 1996
• 20: Zsófia Ruttkay , Han Noot, Animated CharToon faces, Proceedings of the 1st international symposium on Non-photorealistic animation and rendering, p.91-100, June 05-07, 2000, Annecy, France [doi>10.1145/340916.340928]
• 21: Michael P. Salisbury , Michael T. Wong , John F. Hughes , David H. Salesin, Orientable textures for image-based pen-and-ink illustration, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.401-406, August 1997 [doi>10.1145/258734.258890]
• 22: K. Waters and T. M. Levergood. DECface: An automatic lip-synchronization algorithm for synthetic faces. Technical report, DEC Cambridge Research Lab, 1993.
• 23: Y. Xu, B. Guo, and H. Shum. Chaos Mosaic:fase and memory efficient texture synthesis. Technical Report 32, Microsoft Research Technical Report, Apr. 2000.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.3 Picture/Image Generation

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
  I.6 SIMULATION AND MODELING
      I.6.3 Applications

Keywords:
applications, example-based learning, lip-syncing, multi-modal interaction and integration, non-parametric sampling, user interfaces