Emotional gaze behavior generation in human-agent interaction

ABSTRACT

Gaze behavior plays an important role in face to face communication in that it conveys nonverbal information and emotional intent beyond speech. This research proposes a computational framework for generating emotional gaze behavior in a virtual agent, concentrating on analysis and synthesis of primary and intermediate emotions through gaze behavior. We utilize parameters picked from the AU-Coded facial expression database and real-time eye movement data (pupil size, blink rate and saccade) to model primary emotions and describe a rule-based approach to generate intermediate ones.

REFERENCES

• 1: Rosalind W. Picard, Affective computing, MIT Press, Cambridge, MA, 1997
• 2: Moving Pictures Expert Group, ISO/IEC 14496, International Standard on Coding of Audio-Visual Objects
• 3: Gu, E. and Badler, N.I. Visual Attention and Eye Gaze During Multiparty Conversations with Distractions. In Proc. IVA 2007, 193--204
• 4: Brent Lance , Stacy C. Marsella, Emotionally Expressive Head and Body Movement During Gaze Shifts, Proceedings of the 7th international conference on Intelligent Virtual Agents, September 17-19, 2007, Paris, France [doi>10.1007/978-3-540-74997-4_8]
• 5: Brent Lance , Stacy C. Marsella, The Relation between Gaze Behavior and the Attribution of Emotion: An Empirical Study, Proceedings of the 8th international conference on Intelligent Virtual Agents, September 01-03, 2008, Tokyo, Japan [doi>10.1007/978-3-540-85483-8_1]
• 6: Brent J. Lance , Stacy C. Marsella, A model of gaze for the purpose of emotional expression in virtual embodied agents, Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems, May 12-16, 2008, Estoril, Portugal
• 7: Ekman, P. and Friesen, W. V. The Facial Action Coding System (2nd edn), Weidenfeld and Nicolson, London, UK (2002).
• 8: Plutchik, R. Emotion: A Psychoevolutionary Synthesis. Harper and Row, New York, USA (1980).
• 9: Whissel, C.M. The dictionary of affect in language, in Emotion: Theory, Research and Experience. Academic Press (1989), 113--131.
• 10: Diana Arellano , Javier Varona , Francisco J. Perales, Generation and visualization of emotional states in virtual characters, Computer Animation and Virtual Worlds, v.19 n.3-4, p.259-270, September 2008 [doi>10.1002/cav.v19:3/4]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: User-centered design; Evaluation/methodology

General Terms:
Human Factors

Keywords:
emotional expression, gaze behavior, human-agent interaction, virtual agent