Automatic design of a control interface for a synthetic face

ABSTRACT

Getting synthetic faces to display natural facial expressions is essential to enhance the interaction between human users and virtual characters. Yet traditional facial control techniques provide precise but complex sets of control parameters, which are not adapted for non-expert users. In this article, we present a system that generates a simple, 2-Dimensional interface that offers an efficient control over the facial expressions of any synthetic character. The interface generation process relies on the analysis of the deformation of a real human face. The principal geometrical and textural variation patterns of the real face are detected and automatically reorganized onto a low-dimensional space. This control space can then be easily adapted to pilot the deformations of synthetic faces. The resulting virtual character control interface makes it easy to produce varied emotional facial expressions, both extreme and subtle. In addition, the continuous nature of the interface allows the production of coherent temporal sequences of facial animation.

REFERENCES

• 1: B. Abboud, F. Davoine, and M. Dang. Facial expression recognition and synthesis based on an appearance model. Signal Processing: Image Communication, 19(8):723--740, 2004.
• 2: Irene Albrecht , Marc Schröder , Jörg Haber , Hans-Peter Seidel, Mixed feelings: expression of non-basic emotions in a muscle-based talking head, Virtual Reality, v.8 n.4, p.201-212, August 2005 [doi>10.1007/s10055-005-0153-5]
• 3: C. Bradford Barber , David P. Dobkin , Hannu Huhdanpaa, The quickhull algorithm for convex hulls, ACM Transactions on Mathematical Software (TOMS), v.22 n.4, p.469-483, Dec. 1996 [doi>10.1145/235815.235821]
• 4: F. L. Bookstein, Principal Warps: Thin-Plate Splines and the Decomposition of Deformations, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.11 n.6, p.567-585, June 1989 [doi>10.1109/34.24792]
• 5: Timothy F. Cootes , Gareth J. Edwards , Christopher J. Taylor, Active Appearance Models, Proceedings of the 5th European Conference on Computer Vision-Volume II, p.484-498, June 02-06, 1998
• 6: R. Cowie, D.-C. Ellen, S. Savvidou, E. McMahon, M. Sawey, and M. Schrder. Feeltrace: An instrument for recording perceived emotion in real time. Proc. of the ISCA Workshop on Speech and Emotion, pages 19--24, 2000.
• 7: Yangzhou Du , Wenyuan Bi , Tao Wang , Yimin Zhang , Haizhou Ai, Distributing expressional faces in 2-D emotional space, Proceedings of the 6th ACM international conference on Image and video retrieval, p.395-400, July 09-11, 2007, Amsterdam, The Netherlands [doi>10.1145/1282280.1282339]
• 8: Yangzhou Du , Xueyin Lin, Emotional facial expression model building, Pattern Recognition Letters, v.24 n.16, p.2923-2934, December 2003 [doi>10.1016/S0167-8655(03)00153-3]
• 9: P. Ekman, E. Rolls, D. I. Perrett, and H. Ellis. Facial expressions of emotion: An old controversy and new findings. Philosophical Transactions: Biological Sciences, 335(1273):63--69, 1992.
• 10: A. Eleftheriadis, C. Herpel, G. Rajan, and L. Ward. Mpeg-4 systems, text for iso/iec fcd 14496-1 systems. In MPEG-4 SNHC. 1998.
• 11: Changbo Hu , Ya Chang , Rogerio Feris , Matthew Turk, Manifold Based Analysis of Facial Expression, Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'04) Volume 5, p.81, June 27-July 02, 2004
• 12: P. Kalra, A. Mangili, N. Magnenat-Thalmann, and D. Thalmann. Simulation of facial muscle actions based on rational free form deformations. Computer Graphics Forum, 11(3):59--69, 1992.
• 13: C.-S. Lee, A. Elgammal, and D. Metaxas. Synthesis and control of high resolution facial expressions for visual interactions. IEEE International Conference on Multimedia and Expo (ICME), pages 65--68, 2006.
• 14: N. Magnenat-Thalmann, E. Primeau, and D. Thalmann. Abstract muscle action procedures for human face animation. Visual Computer, 3(5):290--7, 1988.
• 15: A. Mehrabian. Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament. Current Psychology, 14(4):261--292, 1996.
• 16: Magalie Ochs , Catherine Pelachaud , David Sadek, An empathic virtual dialog agent to improve human-machine interaction, Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems, May 12-16, 2008, Estoril, Portugal
• 17: F. I. Parke, Parameterized Models for Facial Animation, IEEE Computer Graphics and Applications, v.2 n.9, p.61-68, September 1982 [doi>10.1109/MCG.1982.1674492]
• 18: C. Pelachaud, N. I. Badler, and M. Steedman. Generating facial expressions for speech. Cognitive Science, 20(1):1--46, 1996.
• 19: R. Plutchik. The nature of emotions. American Scientist, 89(4):344, 2001.
• 20: Z. Ruttkay, H. Noot, and P. ten Hagen. Emotion disc and emotion squares: tools to explore the facial expression space. Computer Graphics Forum, 22(1):49--53, 2003.
• 21: H. Schlosberg. The description of facial expressions in terms of two dimensions. Journal of Experimental Psychology, 44(4), 1952.
• 22: C. Shan, S. Gong, and P. W. McOwan. Appearance manifold of facial expressions. In Computer Vision in Human-Computer Interaction, pages 221--230. Springer, Berlin, 2005.
• 23: N. Tsapatsoulis, A. Raouzaiou, S. Kollias, R. Cowie, and D.-C. Ellen. Emotion recognition and synthesis based on mpeg-4 faps. In MPEG-4 Facial Animation -- The standard, implementations and applications, pages 141--167. John Wiley and Sons, Hillsdale, NJ, USA, 2002.
• 24: M.-L. Viaud and H. Yahia. Facial animation with wrinkles. Eurographics Worshop on Animation and Simulation, 1992.
• 25: Keith Waters, A muscle model for animation three-dimensional facial expression, Proceedings of the 14th annual conference on Computer graphics and interactive techniques, p.17-24, August 1987
• 26: C. Whissel. The dictionary of affect in language. In R. Plutchik and H. Kellerman, editors, Emotion: Theory, Research, and Experience, pages 113--131. Academic Press, New York, 1989.
• 27: Shen Zhang , Zhiyong Wu , Helen M. Meng , Lianhong Cai, Facial Expression Synthesis Using PAD Emotional Parameters for a Chinese Expressive Avatar, Proceedings of the 2nd international conference on Affective Computing and Intelligent Interaction, September 12-14, 2007, Lisbon, Portugal [doi>10.1007/978-3-540-74889-2_3]
• 28: Chuan Zhou , Xueyin Lin, Facial expressional image synthesis controlled by emotional parameters, Pattern Recognition Letters, v.26 n.16, p.2611-2627, December 2005 [doi>10.1016/j.patrec.2005.06.007]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Artificial, augmented, and virtual realities

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Animations

General Terms:
Experimentation

Keywords:
aam, avatar, facial animation, traveling salesman problem, user interface, virtual character