Automatic learning and generation of social behavior from collective human gameplay

ABSTRACT

Current approaches to authoring behavior and dialogue for agents that interact with humans in virtual environments are labor intensive, yet often yield less robust results than desired in the face of the incredible variance possible in human input. The growing number of people playing multiplayer games online provides a potentially better alternative to hand-authored content -- capturing behavior and dialogue from human-human interactions, and automating agents with this data. This paper documents promising results from the first iteration of a Collective Artificial Intelligence system that generates behavior and dialogue in real-time from data captured from over 11,000 players of The Restaurant Game. We first describe the game, the collective memory system, and the proposal-critique driven agent architecture, and then demonstrate quantitatively that our system preserves the texture, or meaningful local coherence, of human social interaction.

REFERENCES

• 1: Barto, A. G., Sutton, R. S.,&Anderson, C. W. 1983. Neuronlike elements that can solve difficult learning control problems. IEEE Trans, on Systems, Man, and Cybernetics, 13, 835--846.
• 2: Bratman, M. E. 1987. Intentions, Plans, and Practical Reason. Harvard University Press: Cambridge, MA.
• 3: Burke, R., Isla, D., Downie, M., Ivanov, Y., and Blumberg, B. 2001. CreatureSmarts: The art and architecture of a virtual brain. Proceedings of the Game Developers Conference, 147--166. International Game Developers Association.
• 4: Fikes, R. E. and Nilsson, N. J. 1971. STRIPS: A new approach to the application of theorem proving to problem solving. Artificial Intelligence, 2(3--4), 189--208.
• 5: Fleischman, M. and Roy, D. 2005. Intentional context in situated language learning. Ninth Conference on Computational Natural Language Learning.
• 6: A. L. Gorin , G. Riccardi , J. H. Wright, How may I help you?, Speech Communication, v.23 n.1-2, p.113-127, Oct. 1997 [doi>10.1016/S0167-6393(97)00040-X]
• 7: Gorniak, P. and Roy, D. 2007. Situated language understanding as filtering perceived affordances. Cognitive Science, 31(2), 197--231.
• 8: Gusfield, D. 1997. Algorithms on Strings, Trees and Sequences: Computer Science and Computational Biology. Chapter 6: Linear time construction of suffix trees, 94--121. Cambridge University Press.
• 9: Dana S. Nau , Yue Cao , Amnon Lotem , Hector Muñoz-Avila, SHOP: Simple Hierarchical Ordered Planner, Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, p.968-975, July 31-August 06, 1999
• 10: Orkin, J. and Roy, D. 2007. The Restaurant Game: Learning social behavior and language from thousands of players online. Journal of Game Development, 3(1), 39--60.
• 11: Orkin, J. 2005. Agent architecture considerations for real-time planning in games. Proceedings of Artificial Intelligence&Interactive Digital Entertainment. AAAI Press.
• 12: Kishore Papineni , Salim Roukos , Todd Ward , Wei-Jing Zhu, BLEU: a method for automatic evaluation of machine translation, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1073083.1073135]
• 13: Deepak Ravichandran , Eduard Hovy, Learning surface text patterns for a Question Answering system, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1073083.1073092]
• 14: Singh, P. and Minsky, M. 2003. An architecture for combining ways to think. Proceedings of the International Conference on Knowledge Intensive Multi-Agent Systems.
• 15: Luis von Ahn , Laura Dabbish, Labeling images with a computer game, Proceedings of the SIGCHI conference on Human factors in computing systems, p.319-326, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985733]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.8 Problem Solving, Control Methods, and Search
          Subjects: Plan execution, formation, and generation

General Terms:
Algorithms, Design, Experimentation, Measurement

Keywords:
computer games, dialogue, learning, natural language, social behavior