Expression cloning

ABSTRACT

We present a novel approach to producing facial expression animations for new models. Instead of creating new facial animations from scratch for each new model created, we take advantage of existing animation data in the form of vertex motion vectors. Our method allows animations created by any tools or methods to be easily retargeted to new models. We call this process expression cloning and it provides a new alternative for creating facial animations for character models. Expression cloning makes it meaningful to compile a high-quality facial animation library since this data can be reused for new models. Our method transfers vertex motion vectors from a source face model to a target model having different geometric proportions and mesh structure (vertex number and connectivity). With the aid of an automated heuristic correspondence search, expression cloning typically requires a user to select fewer than ten points in the model. Cloned expression animations preserve the relative motions, dynamics, and character of the original facial animations.

REFERENCES

• 1: Sumit Basu , Nuria Oliver , Alex Pentland, 3D Modeling and Tracking of Human Lip Motions, Proceedings of the Sixth International Conference on Computer Vision, p.337, January 04-07, 1998
• 2: Armin Bruderlin , Lance Williams, Motion signal processing, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.97-104, September 1995 [doi>10.1145/218380.218421]
• 3: M. Cohen, D. Massaro, Modeling Co-articulation in Synthetic Visual Speech. In N. Magnenat-Thalmann, and D. Thalmann Editors, Model and Technique in Computer Animation, 1993, 139-156, Springer-Verlag, Tokyo
• 4: Matthias Eck , Tony DeRose , Tom Duchamp , Hugues Hoppe , Michael Lounsbery , Werner Stuetzle, Multiresolution analysis of arbitrary meshes, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.173-182, September 1995 [doi>10.1145/218380.218440]
• 5: M. Eck, Interpolation Methods for Reconstruction of 3D Surfaces from Sequences of Planar Slices, CAD und Computergraphik, Vol. 13, No. 5, Feb. 1991, 109 - 120
• 6: R. Enciso, J. Li, D. Fidaleo, T-Y. Kim, J-Y.Noh, U. Neumann, Synthesis of 3D Faces, International Workshop on Digital and Computational Video, 2000
• 7: M. Escher , I. Pandzic , N. Magnenat Thalmann, Facial Deformations for MPEG-4, Proceedings of the Computer Animation, p.56, June 08-10, 1998
• 8: D. Fidaleo, J-Y. Noh, T. Kim, R. Enciso, U.Neumann, Classification and Volume Morphing for Performance- Driven Facial Animation, International Workshop on Digital and Computational Video, 2000
• 9: Michael Gleicher, Retargetting motion to new characters, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.33-42, July 1998 [doi>10.1145/280814.280820]
• 10: G.H. Golub, M. Heath, G. Wahba. Generalized Crossvalidation as a Method for Choosing a Good Ridge Parameter. Technometrics, 21(2):215-223, 1979
• 11: Brian Guenter , Cindy Grimm , Daniel Wood , Henrique Malvar , Fredric Pighin, Making faces, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.55-66, July 1998 [doi>10.1145/280814.280822]
• 12: P. Kalra, A. Mangili, N. M. Thalmann, D. Thalmann, Simulation of Facial Muscle Actions Based on Rational Free From Deformations, Eurographics 1992, vol. 11(3) 59-69
• 13: Takashi Kanai , Hiromasa Suzuki , Fumihiko Kimura, Metamorphosis of Arbitrary Triangular Meshes, IEEE Computer Graphics and Applications, v.20 n.2, p.62-75, March 2000 [doi>10.1109/38.824544]
• 14: James R. Kent , Wayne E. Carlson , Richard E. Parent, Shape transformation for polyhedral objects, Proceedings of the 19th annual conference on Computer graphics and interactive techniques, p.47-54, July 1992
• 15: Aaron W. F. Lee , David Dobkin , Wim Sweldens , Peter Schröder, Multiresolution mesh morphing, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.343-350, July 1999 [doi>10.1145/311535.311586]
• 16: Yuencheng Lee , Demetri Terzopoulos , Keith Walters, Realistic modeling for facial animation, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.55-62, September 1995 [doi>10.1145/218380.218407]
• 17: J. P. Lewis , Matt Cordner , Nickson Fong, Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.165-172, July 2000 [doi>10.1145/344779.344862]
• 18: Thomas Maurer , Christoph von der Malsburg, Tracking and Learning Graphs and Pose on Image Sequences of Faces, Proceedings of the 2nd International Conference on Automatic Face and Gesture Recognition (FG '96), p.76, October 14-16, 1996
• 19: J.E. Moody, The Effective Number of Parameters: An Analysis of Generalization and Regularization in Nonlinear Learning Systems, Neural Information Processing Systems 4, 847-854, Morgan Kaufmann, California, 1992
• 20: M. J. L. Orr, Optimizing the Widths of RBFs, Fifth Brazilian Symposium on Neural Networks, Brazil, 1998
• 21: J. Ostermann, Animation of Synthetic Faces in MPEG-4, Proceedings of the Computer Animation, p.49, June 08-10, 1998
• 22: F.I. Parke, Parameterized Models for Facial Animation. IEEE Computer Graphics and Applications, 1982, vol. 2(9) 61 - 68
• 23: Frédéric Pighin , Jamie Hecker , Dani Lischinski , Richard Szeliski , David H. Salesin, Synthesizing realistic facial expressions from photographs, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.75-84, July 1998 [doi>10.1145/280814.280825]
• 24: Stephen M. Platt , Norman I. Badler, Animating facial expressions, ACM SIGGRAPH Computer Graphics, v.15 n.3, p.245-252, August 1981
• 25: William K. Pratt, Digital image processing (2nd ed.), John Wiley & Sons, Inc., New York, NY, 1991
• 26: Franco P. Preparata , Michael I. Shamos, Computational geometry: an introduction, Springer-Verlag New York, Inc., New York, NY, 1985
• 27: Yoshihisa Shinagawa , Tosiyasu L. Kunii, Unconstrained Automatic Image Matching Using Multiresolutional Critical-Point Filters, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.20 n.9, p.994-1010, September 1998 [doi>10.1109/34.713364]
• 28: M. Stone, Toward a Model of Three-Dimensional Tongue Movement. Journal of Phonetics, 1991, Vol. 19, 309-320
• 29: F. Ulgen, A Step Toward Universal Facial Animation via Volume Morphing, 6 th IEEE International Workshop on Robot and Human communication, 1997, 358-363
• 30: K. Waters, J. Frisbie, A Coordinated Muscle Model for Speech Animation, Graphics Interface, 1995, 163 - 170
• 31: Keith Waters, A muscle model for animation three-dimensional facial expression, Proceedings of the 14th annual conference on Computer graphics and interactive techniques, p.17-24, August 1987
• 32: Lance Williams, Performance-driven facial animation, Proceedings of the 17th annual conference on Computer graphics and interactive techniques, p.235-242, September 1990, Dallas, TX, USA

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.9 Robotics
          Subjects: Kinematics and dynamics
  I.3 COMPUTER GRAPHICS
      I.3.3 Picture/Image Generation
          Subjects: Display algorithms
      I.3.5 Computational Geometry and Object Modeling
          Subjects: Geometric algorithms, languages, and systems
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Animation

General Terms:
Algorithms, Design, Theory

Keywords:
deformations, facial animation, morphing, neural nets