Providing expressive gaze to virtual animated characters in interactive applications

ABSTRACT

Eyes play an important role in communication among people. Motions of the eye express emotions and regulate the flow of conversation. Hence we consider fundamental that virtual humans or other characters present convincing and expressive gaze in applications such as Embodied Conversational Agents (ECAs), games and movies. However, we perceive that in many applications that require automatic generation of facial movements, such as ECA, character's eye motion does not carry meaning related to its expressiveness. This work proposes a model for the automatic generation of expressive gaze by examining eye behavior in different affective states. To collect data related to gaze and expressiveness, we looked at Computer Graphics movies. This data was used as a basis to describe gaze expressions in the proposed model. We also implemented a prototype and performed some tests with users in order to observe the impact of eye behavior during some expressions of emotion. The results show that the model is capable of generating eye motions that are coherent with the affective states of the virtual character.

REFERENCES

• 1: Koray Balci, Xface: MPEG-4 based open source toolkit for 3D Facial Animation, Proceedings of the working conference on Advanced visual interfaces, May 25-28, 2004, Gallipoli, Italy [doi>10.1145/989863.989935]
• 2: Balci, K. 2007. Xface. http://xface.itc.it/. Accessed June 2007.
• 3: Brooker, D. 2007. Body language -- eyes. http://www.hash.com/users/threechiken/eyel.html. Accessed April 2007.
• 4: Cassell, J., Torres, O., and Prevost, S. 1997. Modeling gaze behavior as a function of discourse structure. In Proceedings of the First International Workshop on Human-Computer Conversations.
• 5: Cohen, M. F., Colburn, R. A., and Drucker, S. M. 2000. The role of eye gaze in avatar mediated conversational interfaces. Tech. Rep. MSR-TR-2000--81, Microsoft Corp.
• 6: Jeffrey F. Cohn, Foundations of human computing: facial expression and emotion, Proceedings of the 8th international conference on Multimodal interfaces, November 02-04, 2006, Banff, Alberta, Canada [doi>10.1145/1180995.1181043]
• 7: Courty, N., Breton, G., and Pelé, D. 2003. Embodied in a look: Bridging the gap between humans and avatars. In Proceedings of the 4th International Workshop on Intelligent Virtual Agents (IVA), 111--118.
• 8: Ekman, P. 1993. Facial expression and emotion. American Psychologist 46, 9 (April), 384--392.
• 9: Clark Davidson Elliott, The affective reasoner: a process model of emotions in a multi-agent system, Northwestern University, Evanston, IL, 1992
• 10: Atsushi Fukayama , Takehiko Ohno , Naoki Mukawa , Minako Sawaki , Norihiro Hagita, Messages embedded in gaze of interface agents --- impression management with agent's gaze, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503385]
• 11: Gu, E. and Badler, N. I. 2006. Visual attention and eye gaze during multiparty conversations with distractions. In Proceedings of the International Workshop on Intelligent Virtual Agents (IVA), 193--204.
• 12: Itti, L., Dhavale, N., and Pighin, F. 2004. Realistic avatar eye and head animation using a neurobiological model of visual attention. SPIE Applications and Science of Neural Networks, Fuzzy Systems, and Evolutionary Computation, VI 5200, 64--78.
• 13: Roberto Ierusalimschy, Programming in Lua, Second Edition, Lua.Org, 2006
• 14: Lance, B., Stacy, M., and Koizumi, D. 2004. Towards expressive gaze manner in embodied virtual agents. In Autonomous Agents and Multi-Agent Systems Workshop on Empathic Agents.
• 15: Sooha Park Lee , Jeremy B. Badler , Norman I. Badler, Eyes alive, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 16: Ortony, A. and Turner, T. 1990. What's basic about basic emotions? Psychological Rev. 97, 315--331.
• 17: Osfield, R. 2007. Open scene graph. http://www.openscenegraph.org/. Accessed June 2007.
• 18: J. Ostermann, Animation of Synthetic Faces in MPEG-4, Proceedings of the Computer Animation, p.49, June 08-10, 1998
• 19: Rosalind W. Picard, Affective computing, MIT Press, Cambridge, MA, 1997
• 20: Poggi, I. and Pelachaud, C. 2002. Signals and meanings of gaze in animated faces. In Language, Vision and Music, P. McKevitt et al. (eds.), 133--144.
• 21: Isabella Poggi , Catherine Pelachaud , Fiorella De Rosis, Eye communication in a conversational 3D synthetic agent, AI Communications, v.13 n.3, p.169-181, August 2000
• 22: Rodrigues, P. S., Queiroz, R. B., Barros, L., Feijó, B., Velho, L., and Musse, S. R. 2007. Automatically generating eye motion in virtual agents. In Proceedings of the IX Symposium on Virtual and Augmented Reality (SVR2007), 84--91.
• 23: Bjarne Stroustrup, The C++ Programming Language, Third Edition, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1997
• 24: Roel Vertegaal , Robert Slagter , Gerrit van der Veer , Anton Nijholt, Eye gaze patterns in conversations: there is more to conversational agents than meets the eyes, Proceedings of the SIGCHI conference on Human factors in computing systems, p.301-308, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365119]
• 25: Vinayagamoorthy, V., Garau, M., Steed, A., And Slater, M. 2004. An eye gaze model for dyadic interaction in an immersive virtual environment: Practice and experience. Computer Graphics Forum 23.
• 26: Mason Woo , Jackie Neider , Tom Davis , Dave Shreiner, OpenGL Programming Guide: The Official Guide to Learning OpenGL, Version 1.2, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1999

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Animation

General Terms:
Algorithms, Design, Experimentation, Human Factors

Keywords:
Eye animation, behavioral animation, expressiveness, facial animation, gaze behavior