Facial animation framework for the web and mobile platforms

ABSTRACT

Talking virtual characters are graphical simulations of real or imaginary persons capable of human-like behavior, most importantly talking and gesturing. They may find applications on the Internet and mobile platforms as newscasters, customer service representatives, sales representatives, guides etc. After briefly discussing the possible applications and the technical requirements for bringing such applications to life, we describe our approach to enable these applications: the Facial Animation Framework. This framework consists of (1) a lightweitht, portable, MPEG-4 compatible Facial Animation Player, (2) a system for fast production of ready-to-animate, MPEG-4 compatible face models and (3) a plethora of MPEG-4 compatible tools for Facial Animation content production. We believe that this kind of approach offers enough flexibility to rapidly adapt to a broad range of applications involving facial animation on various platforms.

REFERENCES

• 1: Jörgen Ahlberg, Using the Active Appearance Algorithm for Face and Facial Feature Tracking, Proceedings of the IEEE ICCV Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems (RATFG-RTS'01), p.68, July 13-August 13, 2001
• 2: A.L.I.C.E. natural language A.I. parser and chat robot, www.alicebot.org
• 3: Kiyoshi Arai, Tsuneya Kurihara, Ken-ichi Anjyo, "Bilinear interpolation for facial expressions and methamrphosis in real-time animation", The Visual Computer, 12:105--116, 1996.
• 4: M.M.Cohen and D.W.Massaro, "Modeling Coarticulation in Synthetic Visual Speech." In M.Thalmann & D.Thalmann (Eds.) Computer Animation'93. Tokyo: Springer-Verlag.
• 5: E. Cosatto , H. Graf, Sample-Based Synthesis of Photo-Realistic Talking Heads, Proceedings of the Computer Animation, p.103, June 08-10, 1998
• 6: J. E. Chadwick , D. R. Haumann , R. E. Parent, Layered construction for deformable animated characters, ACM SIGGRAPH Computer Graphics, v.23 n.3, p.243-252, July 1989
• 7: P. Eisert, S. Chaudhuri and B. Girod, "Speech Driven Synthesis of Talking Head Sequences," 3D Image Analysis and Synthesis, pp. 51--56, Erlangen, November 1997.
• 8: M. Escher , I. Pandzic , N. Magnenat Thalmann, Facial Deformations for MPEG-4, Proceedings of the Computer Animation, p.56, June 08-10, 1998
• 9: Robert Forchheimer and Olov Fahlander, "Low Bit-rate Coding through Animation", Proceedings Picture Coding Symposium 83.
• 10: Robert Forchheimer, Olov Fahlander and Torbj¿rn Kronander, "A Semantic Approach to the Transmission of Face Images," Proceedings Picture Coding Symposium 84.
• 11: ISO/IEC 14496 - MPEG-4 International Standard, Moving Picture Experts Group, www.cselt.it/mpeg
• 12: Kalra P., Mangili A., Magnenat-Thalmann N., Thalmann D., Simulation of Facial Muscle Actions based on Rational Free Form Deformation", Proceedings Eurographics 92, pp. 65--69.
• 13: N. Magnenat-Thalmann, N.E. Primeau, D. Thalmann, "Abstract muscle actions procedures for human face animation", Visual Computer, 3(5):290--297, 1988.
• 14: Jun-yong Noh , Ulrich Neumann, Expression cloning, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.277-288, August 2001 [doi>10.1145/383259.383290]
• 15: Igor S. Pandzic, Joern Ostermann, David Millen, "Synthetic Faces: What are they good for?", The Visual Computer, 1999.
• 16: Igor S. Pandzic, Gael Sannier, "From Photographs to Interactive Virtual Characters on the Web", Proc. Scanning 2000, Paris, France.
• 17: Igor S. Pandzic, "Life on the Web", Software Focus Journal, 2(2):52--59, John Wiley & Sons, 2001.
• 18: I.S. Pandzic,"A Web-Based MPEG-4 Facial Animation System", Proc. ICAV 3D 2001, demonstration at www.icg.isy.liu.se/~igor/MpegWeb
• 19: Frederic Ira Parke, A parametric model for human faces., 1974
• 20: F.I. Parke, "Parametrized models for facial animation", IEEE Computer Graphics and Applications, 2(9):61--68, November 1982.
• 21: Frederic I. Parke , Keith Waters, Computer facial animation, A. K. Peters, Ltd., Natick, MA, 1996
• 22: Pearson, "Development in Model-Based Video Coding", Proc. of the IEEE, 83(6):892--906, June 1995.
• 23: Stephen M. Platt , Norman I. Badler, Animating facial expressions, ACM SIGGRAPH Computer Graphics, v.15 n.3, p.245-252, August 1981
• 24: Quartz Version 6.0, Symbian Technical Paper, Symbian Developer Network, www.symbiandevnet.com/techlib/techcomms/techpapers/papers/v6/over/quartz/index.html
• 25: Shout 3D, Eyematic Interfaces Incorporated, http://www.shout3d.com/
• 26: VRML, ISO/IEC 14772-1:1999, www.web3d.org/fs_specifications.htm
• 27: Tekalp M.A., Ostermann J., "Face and 2-D Mesh Animation in MPEG-4", Image Communication Journal, Tutorial Issue on MPEG-4 Standard, Elsevier, 2000.
• 28: D. Terzopoulos, K. Waters, "Physically-based facial modeling, analysis and animation", Journal of Visualization and Computer Animation, 1(4):73--80, 1990.
• 29: Joseph Weizenbaum, ELIZA—a computer program for the study of natural language communication between man and machine, Communications of the ACM, v.9 n.1, p.36-45, Jan. 1966 [doi>10.1145/365153.365168]
• 30: Keith Waters, A muscle model for animation three-dimensional facial expression, ACM SIGGRAPH Computer Graphics, v.21 n.4, p.17-24, July 1987
• 31: W Interactive SARL, www.winteractive.fr

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Animations

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Artificial, augmented, and virtual realities
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Natural language; User-centered design; Graphical user interfaces (GUI); Auditory (non-speech) feedback

General Terms:
Algorithms, Design, Human Factors

Keywords:
FBA, MPEG-4, VRML, facial animation, facial motion cloning, talking head, virtual characters, virtual humans, visual text-to-speech