User's gestural exploration of different virtual agents' expressive profiles

ABSTRACT

Designing affective user interfaces involving expressive characters raises several questions. The system should be able to display facial expressions of complex emotions as dynamic and realtime reactions to user's inputs. From a cognitive point of view, designers need to know how the user will perceive the dynamics of these facial expressions as a function of his/her input. We aim at evaluating if users can perceive different expressive profiles of a virtual character by manually controlling its expressions and observing its reaction to his/her input. This paper describes our platform that enables a virtual character to display blended facial expressions of emotions as realtime continuous reactions to users' gesture input. We explain the techniques underlying the computation of intermediate facial expressions of emotion, and their control in the 3D space PAD (Pleasure, Arousal, Dominance) using gesture input. Preliminary results of a perceptive study show the potential of such an approach for assessing the dynamics of the perception of emotional expressions during gesture interaction with virtual characters endowed with different expressive profiles.

REFERENCES

• 1: Frank, M. G., Ekman, P. and Friesen, W. V. Behavioral markers and recognizability of the smile of enjoyment. Journal of Personality and Social Psychology, 64 (1993), 83--93.
• 2: Bailly, G., Bérar, M., Elisei, F. and Odisi, M. Audiovisual Speech Synthesis. International Journal of Speech Technology. Special Issue on Speech Synthesis: Part II., 6, 4 (2003), 331--346.
• 3: Ekman, P. and Friesen, W. V. Unmasking the face. A guide to recognizing emotions from facial clues. Prentice-Hall Inc., Englewood Cliffs, N.J., 1975.
• 4: Niewiadomski, R. A model of complex facial expressions in interpersonal relations for animated agents. PhD. dissertation, University of Perugia, 2007.
• 5: Tsapatsoulis, N., Raouzaiou, A., Kollias, S., Cowie, R. and Douglas-Cowie, E. Emotion Recognition and Synthesis based on MPEG-4 FAPs. MPEG-4 Facial Animation. John Wiley & Sons, 141--167, UK, 2002.
• 6: Irene Albrecht , Marc Schröder , Jörg Haber , Hans-Peter Seidel, Mixed feelings: expression of non-basic emotions in a muscle-based talking head, Virtual Reality, v.8 n.4, p.201-212, August 2005 [doi>10.1007/s10055-005-0153-5]
• 7: Robert Forchheimer , Igor S. Pandzic , Algirdas Pakstas, MPEG-4 Facial Animation: The Standard,Implementation and Applications, John Wiley & Sons, Inc., New York, NY, 2002
• 8: L. Malatesta , A. Raouzaiou , K. Karpouzis , S. Kollias, MPEG-4 facial expression synthesis, Personal and Ubiquitous Computing, v.13 n.1, p.77-83, January 2009 [doi>10.1007/s00779-007-0164-1]
• 9: Buisine, S., Abrilian, S., Niewiadomski, R., Martin, J.-C., Devillers, L. and Pelachaud, C. 2006. Perception of Blended Emotions: from Video Corpus to Expressive Agent. In Proceedings of the 6th International Conference on Intelligent Virtual Agents (IVA'2006) (Marina del Rey, USA, 21-23 august, 2006). Springer.
• 10: Patrick Gebhard, ALMA: a layered model of affect, Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems, July 25-29, 2005, The Netherlands [doi>10.1145/1082473.1082478]
• 11: Sumedha Kshirsagar, A multilayer personality model, Proceedings of the 2nd international symposium on Smart graphics, p.107-115, June 11-13, 2002, Hawthorne, New York [doi>10.1145/569005.569021]
• 12: D'Alessandro, C., D'Alessandro, N., Le Beux, S., Simko, J., Cetin, F. and Pirker, H. 2005. The speech conductor: gestural control of speech synthesis. In Proceedings of the eNTERFACE 2005. The SIMILAR NoE Summer Workshop on Multimodal Interfaces (Mons, Belgium, July 18-August 12, 2005).
• 13: Christian Jacquemin, Pogany: A Tangible Cephalomorphic Interface for Expressive Facial Animation, Proceedings of the 2nd international conference on Affective Computing and Intelligent Interaction, September 12-14, 2007, Lisbon, Portugal [doi>10.1007/978-3-540-74889-2_49]
• 14: Ana Paiva , Gerd Andersson , Kristina Höök , Dário Mourão , Marco Costa , Carlos Martinho, SenToy in FantasyA: Designing an Affective Sympathetic Interface to a Computer Game, Personal and Ubiquitous Computing, v.6 n.5-6, p.378-389, December 2002 [doi>10.1007/s007790200043]
• 15: Mehrabian, A. Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament. Current Psychology, 14 (1996), 261--292.
• 16: Becker, C., Leßmann, N., Kopp, S. and Wachsmuth, I. 2006. Connecting feelings and thoughts - modeling the interaction of emotion and cognition in embodied agents. In Proceedings of the Seventh International Conference on Cognitive Modeling (ICCM-06) (Trieste, Italy, 2006). Edizioni Goliardiche.
• 17: Jean-Claude Martin , Christophe D'Alessandro , Christian Jacquemin , Brian Katz , Aurélien Max , Laurent Pointal , Albert Rilliard, 3D Audiovisual Rendering and Real-Time Interactive Control of Expressivity in a Talking Head, Proceedings of the 7th international conference on Intelligent Virtual Agents, September 17-19, 2007, Paris, France [doi>10.1007/978-3-540-74997-4_3]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

General Terms:
Algorithms, Design, Human Factors

Keywords:
expressive agent, facial expressions, realtime interaction