Interacting with virtual characters in interactive storytelling

ABSTRACT

In recent years, several paradigms have emerged for interactive storytelling. In character-based storytelling, plot generation is based on the behaviour of autonomous characters. In this paper, we describe user interaction in a fully-implemented prototype of an interactive storytelling system. We describe the planning techniques used to control autonomous characters, which derive from HTN planning. The hierarchical task network representing a characters' potential behaviour constitute a target for user intervention, both in terms of narrative goals and in terms of physical actions carried out on stage. We introduce two different mechanisms for user interaction: direct physical interaction with virtual objects and interaction with synthetic characters through speech understanding. Physical intervention exists for the user in on-stage interaction through an invisible avatar: this enables him to remove or displace objects of narrative significance that are resources for character's actions, thus causing these actions to fail. Through linguistic intervention, the user can influence the autonomous characters in various ways, by providing them with information that will solve some of their narrative goals, instructing them to take direct action, or giving advice on the most appropriate behaviour. We illustrate these functionalities with examples of system-generated behaviour and conclude with a discussion of scalability issues.

REFERENCES

• 1: Ryohei Nakatsu , Naoko Tosa, Interactive movies, Handbook of Internet and multimedia: systems and applications, CRC Press, Inc., Boca Raton, FL, 1999
• 2: R. Hill , J. Gratch , W. L. Johnson , C. Kyriakakis , C. LaBore , R. Lindheim , S. Marsella , D. Miraglia , B. Moore , J. Morie , J. Rickel , M. Thiébaux , L. Tuch , R. Whitney , J. Douglas , W. Swartout, Toward the holodeck: integrating graphics, sound, character and story, Proceedings of the fifth international conference on Autonomous agents, p.409-416, May 2001, Montreal, Quebec, Canada [doi>10.1145/375735.376390]
• 3: Aylett R., 1999. Narrative in Virtual Environments - Towards Emergent Narrative. In: AAAI 1999 Fall Symposium on Narrative Intelligence.
• 4: Dautenhahn, K., 1998. Story-Telling in Virtual Environments, ECAI'98 Workshop on Intelligent Virtual Environments, Brighton, UK.
• 5: Isabel Machado , Ana Paiva , Paul Brna, Real Characters in Virtual Stories, Proceedings of the International Conference on Virtual Storytelling: Using Virtual Reality Technologies for Storytelling, p.127-134, September 27-28, 2001
• 6: Marina Umaschi Bers , Justine Cassell, Interactive storytelling systems for children: using technology to explore language and identity, Journal of Interactive Learning Research, v.9 n.2, p.183-215, Dec. 1999
• 7: Sgouros, N.M., Papakonstantinou, G. and Tsanakas, P., 1996. A Framework for Plot Control in Interactive Story Systems, Proceedings AAAI'96, AAAI Press.
• 8: Grabson and Braun, 2001. A Morphological Approach to Interactive Storytelling. Proceedings of Cast01, Living in Mixed Realities, pp. 337--340.
• 9: Barbara Hayes-Roth , Robert van Gent , Daniel Huber, Acting in Character, Creating Personalities for Synthetic Actors, Towards Autonomous Personality Agents, p.92-112, January 1997
• 10: Bickmore, T., and Cassell, J., 1999. Small Talk and Conversational Storytelling in Embodied Interface Agents. AAAI Fall Symposium, "Narrative Intelligence," November 5-7, Cape Cod, MA.
• 11: Mateas, M. and Stern, A., 2000. Socially Intelligent Agents: The Human in the Loop. AAAI Fall Symposium.
• 12: Young, R.M., 2000. Creating Interactive Narrative Structures: The Potential for AI Approaches. AAAI Spring Symposium in Artificial Intelligence and Interactive Entertainment, AAAI Press.
• 13: Young, R.M., 2001. An Overview of the Mimesis Architecture: Integrating Narrative Control into a Gaming Environment. AAAI Spring Symposium in Artificial Intelligence and Interactive Entertainment, AAAI Press.
• 14: Cavazza M, Charles, F. and Mead, S.J., 2001. AI-based Animation for Interactive Storytelling. Proceedings of Computer Animation, IEEE Computer Society Press.
• 15: Victor DeLeon , Robert Berry, Bringing VR to the Desktop: Are You Game?, IEEE MultiMedia, v.7 n.2, p.68-72, April 2000 [doi>10.1109/93.848433]
• 16: Propp, V., 1968. Morphology of the Folktale. University of Texas Press: Austin and London.
• 17: Jay Douglas , Jonathan Gratch, Adaptive Narrative: How Autonomous Agents, Hollywood, and Multiprocessing Operating Systems Can Live Happily Ever After, Proceedings of the International Conference on Virtual Storytelling: Using Virtual Reality Technologies for Storytelling, p.100-112, September 27-28, 2001
• 18: Mateas, M. and Stern, A., 2000. Towards Integrating Plot and Character for Interactive Drama. In Socially Intelligent Agents: The Human in the Loop. Papers from the 2000 AAAI Fall Symposium, Technical Report FS-00-04, AAAI Press, 113--118.
• 19: Bonnie Webber , Norman Badler , Barbara Di Eugenio , Chris Geib , Libby Levison , Michael Moore, Instructions, intentions and expectations, Artificial Intelligence, v.73 n.1-2, p.253-269, Feb. 1995 [doi>10.1016/0004-3702(94)00013-Q]
• 20: Geib, C., 1994. The intentional planning system: Itplans. Proceedings of the 2nd Artificial Intelligence Planning Systems Conference, pp. 55--64.
• 21: Subbarao Kambhampati , Amol Mali , Biplav Srivastava, Hybrid planning for partially hierarchical domains, Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence, p.882-888, July 1998, Madison, Wisconsin, United States
• 22: Subbarao Kambhampati , James A. Hendler, A validation-structure-based theory of plan modification and reuse, Artificial Intelligence, v.55 n.2-3, p.193-258, June 1992 [doi>10.1016/0004-3702(92)90056-4]
• 23: Dana S. Nau , Stephen J. J. Smith , Kutluhan Erol, Control strategies in HTN planning: theory versus practice, Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence, p.1127-1133, July 1998, Madison, Wisconsin, United States
• 24: Smith, S.J.J., Nau, D.S., and Throop, T., 1998. Computer Bridge: A Big Win for AI Planning. AI Magazine, 19(2):93--105.
• 25: Peter William Weyhrauch , Joseph Bates, Guiding interactive drama, 1997
• 26: Geib, C. and Webber, B., 1993. A consequence of incorporating intentions in means-end planning. Working Notes - AAAI Spring Symposium Series: Foundations of Automatic Planning: The Classical Approach and Beyond. AAAI Press.
• 27: Marc Cavazza , Fred Charles , Steven J. Mead, Emergent situations in interactive storytelling, Proceedings of the 2002 ACM symposium on Applied computing, March 11-14, 2002, Madrid, Spain [doi>10.1145/508791.509003]
• 28: Barthes, R. 1966. Introduction a l'Analyse Structurale des Récits (in French), Communications, 8, pp. 1--27.
• 29: Traum, D. and Rickel, J., 2001. Embodied Agents for Multi-party Dialogue in Immersive Virtual Worlds. IJCAI 2001 Workshop on Representation and Reasoning in Practical Dialogue Systems, Seattle, USA.
• 30: Rama Bindiganavale , William Schuler , Jan M. Allbeck , Norman I. Badler , Aravind K. Joshi , Martha Palmer, Dynamically altering agent behaviors using natural language instructions, Proceedings of the fourth international conference on Autonomous agents, p.293-300, June 03-07, 2000, Barcelona, Spain [doi>10.1145/336595.337503]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.8 Problem Solving, Control Methods, and Search
          Subjects: Graph and tree search strategies

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.8 Problem Solving, Control Methods, and Search
          Subjects: Plan execution, formation, and generation; Heuristic methods

General Terms:
Algorithms, Design, Experimentation, Theory, Verification

Keywords:
computer games, interactive storytelling, planning, speech understanding, synthetic characters