ERIC: a generic rule-based framework for an affective embodied commentary agent

ABSTRACT

We present ERIC, an affective embodied agent for realtime commentary in many domains. The underlying architecture is rule-based, generic, and lightweight - based on Java/Jess modules. Apart from reasoning about dynamically changing events, the system can produce coherent natural language and non-verbal behaviour, based on a layered model of affect (personality, mood, emotion). We show how reasoning, template-based natural language generation and affective appraisal can be implemented within the same rule-based paradigm. To make the system domain independent we worked on two different domains, a virtual horse race and a multi-player tank battle game. We empirically evaluated the genericness of the system by measuring the effort it takes to change the domain, and discuss the results.

REFERENCES

• 1: Elisabeth André , Gerd Herzog , Thomas Rist, Generating Multimedia Presentations for RoboCup Soccer Games, RoboCup-97: Robot Soccer World Cup I, p.200-215, January 1998
• 2: Justine Cassell , Tim Bickmore , Lee Campbell , Hannes Vilhjálmsson , Hao Yan, Human conversation as a system framework: designing embodied conversational agents, Embodied conversational agents, MIT Press, Cambridge, MA, 2001
• 3: Patrick Gebhard, ALMA: a layered model of affect, Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems, July 25-29, 2005, The Netherlands [doi>10.1145/1082473.1082478]
• 4: P. Gebhard, M. Klesen, and T. Rist. Coloring multi-character conversations through the expression of emotions. In roceedings of the Tutorial and Research Workshop on Affective Dialogue Systems (ADS'04), pages 128--141, 2004.
• 5: L. R. Goldberg. An alternative "description of personality": The Big-Five factor structure. Journal of Personality and Social Psychology, 59(6):1216--1229, 1990.
• 6: Barbara J. Grosz , Scott Weinstein , Aravind K. Joshi, Centering: a framework for modeling the local coherence of discourse, Computational Linguistics, v.21 n.2, p.203-225, June 1995
• 7: G. Herzog and P. Wazinski. VIsual TRAnslator: Linking perceptions and natural language descriptions. Artificial Intelligence Review, 8(2--3):175--187, 1994.
• 8: Ernest Friedman Hill, Jess in Action: Java Rule-Based Systems, Manning Publications Co., Greenwich, CT, 2003
• 9: M. Kipp, K. H. Kipp, A. Ndiaye, and P. Gebhard. Evaluating the Tangible Interface and Virtual Characters in the Interactive COHIBIT Exhibit. In Proceedings of the 6th International Conference on Intelligent Virtual Agents (IVA 2006). Springer, 2006.
• 10: Michael Kipp , Michael Neff , Kerstin H. Kipp , Irene Albrecht, Towards Natural Gesture Synthesis: Evaluating Gesture Units in a Data-Driven Approach to Gesture Synthesis, Proceedings of the 7th international conference on Intelligent Virtual Agents, September 17-19, 2007, Paris, France [doi>10.1007/978-3-540-74997-4_2]
• 11: M. Klesen, M. Kipp, P. Gebhard, and T. Rist. Staging exhibitions: Methods and tools for modeling narrative structure to produce interactive performances with virtual actors. Virtual Reality. Special Issue on Storytelling in Virtual Environments, 7(1):17--29, 2003.
• 12: François L. A. Knoppel , Almer S. Tigelaar , Danny Oude Bos , Thijs Alofs , Zsófia Ruttkay, Trackside DEIRA: a dynamic engaging intelligent reporter agent, Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems, May 12-16, 2008, Estoril, Portugal
• 13: J.-C. Martin. On the use of the multimodal clues in human behaviour for the modelling of agent co-operative behaviour. Connection Science, Special Issue on "Agent Autonomy in Groups", 14(4):297--309, 2002.
• 14: Mark Maybury , Oliviero Stock , Wolfgang Wahlster, Intelligent Interactive Entertainment Grand Challenges, IEEE Intelligent Systems, v.21 n.5, p.14-18, September 2006 [doi>10.1109/MIS.2006.98]
• 15: H. McBreen and M. Jack. Empirical evaluation of animated agents in a multi-modal retail application. In Proceedings AAAI Fall Symposium: Socially Intelligent Agents, pages 122--126, 2000.
• 16: A. Mehrabian. Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament. Current Psychology, 14(4):261--292, December 1996.
• 17: G. P. Morgan, F. E. Ritter, W. E. Stevenson, and I. N. Schenck. dTank: An environment for architectural comparisons of competitive agents. In L. Allender and T. Kelley, editors, Proceedings of the 14th Conference on Behavior Representation in Modeling and Simulation, 05-BRIMS-043, pages 133--140, Orlando, FL, 2005. University of Central Florida, University of Central Florida.
• 18: A. Ndiaye, P. Gebhard, M. Kipp, M. Klesen, M. Schneider, and W. Wahlster. Ambient intelligence in edutainment: Tangible interaction with life-like exhibit guides. In Proceedings of the first Confernce on INtelligent TEchnologies for interactive enterTAINment (INTETAIN), pages 104--113, Berlin, Heidelberg, 2005. Springer.
• 19: Michael Neff , Michael Kipp , Irene Albrecht , Hans-Peter Seidel, Gesture modeling and animation based on a probabilistic re-creation of speaker style, ACM Transactions on Graphics (TOG), v.27 n.1, p.1-24, March 2008 [doi>10.1145/1330511.1330516]
• 20: Tsukasa Noma , Liwei Zhao , Norman I. Badler, Design of a Virtual Human Presenter, IEEE Computer Graphics and Applications, v.20 n.4, p.79-85, July 2000 [doi>10.1109/38.851755]
• 21: A. Ortony, A. Collins, and G. L. Clore. The Cognitive Structure of Emotions. Cambridge University Press, Cambridge, UK, 1988.
• 22: Rosalind W. Picard, Affective computing, MIT Press, Cambridge, MA, 1997
• 23: J. Rickel and W. L. Johnson. Animated agents for procedural training in virtual reality: Perception, cognition, and motor control. Applied Artificial Intelligence, 13:343--382, 1999.
• 24: T. Rist, E. André, S. Baldes, P. Gebhard, M. Klesen, M. Kipp, P. Rist, and M. Schmitt. A review of the development of embodied presentation agents and their application fields. In H. Prendinger and M. Ishizuka, editors, Life-Like Characters - Tools, Affective Functions, and Applications, pages 377--404. Springer, Heidelberg, 2003.
• 25: C. Rocchi , O. Stock , M. Zancanaro , M. Kruppa , A. Krüger, The museum visit: generating seamless personalized presentations on multiple devices, Proceedings of the 9th international conference on Intelligent user interfaces, January 13-16, 2004, Funchal, Madeira, Portugal [doi>10.1145/964442.964517]
• 26: M. Strauss. Realtime generation of multimodal affective sports commentary for embodied agents. Master's thesis, Saarland University, 2007.
• 27: M. Theune , E. Klabbers , J. R. De Pijper , E. Krahmer , J. Odijk, From data to speech: a general approach, Natural Language Engineering, v.7 n.1, p.47-86, March 2001
• 28: Kees Van Deemter , Emiel Krahmer , Mariët Theune, Real versus Template-Based Natural Language Generation: A False Opposition?, Computational Linguistics, v.31 n.1, p.15-24, March 2005 [doi>10.1162/0891201053630291]
• 29: Hans van Vliet, Software engineering (2nd ed.): principles and practice, John Wiley & Sons, Inc., New York, NY, 2000
• 30: V. Vinayagamoorthy, M. Gillies, A. Steed, E. Tanguy, X. Pan, C. Loscos, and M. Slater. Building Expression into Virtual Characters. In Eurographics Conference State of the Art Report, Vienna, 2006.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.1 Applications and Expert Systems

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.0 General
          Subjects: Cognitive simulation
      I.2.7 Natural Language Processing
          Subjects: Language generation

General Terms:
Design, Human Factors

Keywords:
affect, commentary agents, discourse coherence, embodied characters, embodied conversational agents, event recognition, natural language generation