More than just a pretty face: affordances of embodiment

ABSTRACT

Prior research into embodied interface agents has found that users like them and find them engaging. In this paper, we argue that embodiment can serve an even stronger function if system designers use actual human conversational protocols in the design of the interface. Communicative behaviors such as salutations and farewells, conversational turn-taking with interruptions, and referring to objects using pointing gestures are examples of protocols that all native speakers of a language already know how to perform and that can thus be leveraged in an intelligent interface. We discuss how these protocols are integrated into Rea, an embodied, multi-modal conversational interface agent who acts as a real-estate salesperson, and we show why embodiment is required for their successful implementation.

REFERENCES

• 1: Elisabeth André , Thomas Rist , Jochen Müller, Integrating reactive and scripted behaviors in a life-like presentation agent, Proceedings of the second international conference on Autonomous agents, p.261-268, May 10-13, 1998, Minneapolis, Minnesota, United States [doi>10.1145/280765.280842]
• 2: Azarbayejani, A., Wren, C. and Pentland A. Real-time 3-D tracking of the human body. In Proceedings of IMAGE'COM 96, (Bordeaux, France, May 1996).
• 3: Gene Ball , Dan Ling , David Kurlander , John Miller , David Pugh , Tim Skelly , Andy Stankosky , David Thiel , Maarten Van Dantzich , Trace Wax, Lifelike computer characters: the persona project at Microsoft, Software agents, MIT Press, Cambridge, MA, 1997
• 4: Boyle, E., Anderson, A., and Newlands, A. The Effects of Visibility in a Cooperative Problem Solving Task. Language and Speech 37(l), 1994. l-20.
• 5: Cassell, J. "Embodied Conversation: Integrating Face and Gesture into Automatic Spoken Dialogue Systems." In Luperfoy (ed.), Spoken Dialogue Systems. (to appear) Cambridge, MA: MIT Press.
• 6: J. Cassell , T. Bickmore , M. Billinghurst , L. Campbell , K. Chang , H. Vilhjálmsson , H. Yan, Embodiment in conversational interfaces: Rea, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.520-527, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303150]
• 7: Justine Cassell , Tim Bickmore , Lee Campbell , Hannes Vilhjálmsson , Hao Yan, Human conversation as a system framework: designing embodied conversational agents, Embodied conversational agents, MIT Press, Cambridge, MA, 2001
• 8: Justine Cassell , Catherine Pelachaud , Norman Badler , Mark Steedman , Brett Achorn , Tripp Becket , Brett Douville , Scott Prevost , Matthew Stone, Animated conversation: rule-based generation of facial expression, gesture & spoken intonation for multiple conversational agents, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.413-420, July 1994 [doi>10.1145/192161.192272]
• 9: Cassell, J. and Thbrisson, K. The Power of a Nod and a Glance: Envelope vs. Emotional Feedback in Animated Conversational Agents. Journal of Applied Art@ial Intelligence, in press.
• 10: Cassell, J., Torres, 0. and Prevost, S. Turn taking vs. Discourse Structure: how best to model multimodal conversation. In Wilks (ed.) Machine Conversations. Kluwer, The Hague, 1998.
• 11: Chovil, N. Discourse-Oriented Facial Displays in Conversation. Research on Language and Social Interaction, 25, 163-I 94, 1992.
• 12: CLIPS Reference Manual Version 6.0. Technical Report, Number JSC-25012, Software Technology Branch, Lyndon B. Johnson Space Center, Houston, TX, 1994.
• 13: Tim Finin , Richard Fritzson , Don McKay , Robin McEntire, KQML as an agent communication language, Proceedings of the third international conference on Information and knowledge management, p.456-463, November 29-December 02, 1994, Gaithersburg, Maryland, United States [doi>10.1145/191246.191322]
• 14: Sara Kiesler , Lee Sproull, “Social” human-computer interaction, Human values and the design of computer technology, Center for the Study of Language and Information, Stanford, CA, 1997
• 15: Kendon, A. Conducting Interaction: Patterns of behavior in focused encounters. Cambridge University Press. New York. 1990.
• 16: Koda, T., and Maes, P. Agents with faces: The effect of personification. Proceedings of the F$h IEEE International Workshop on Robot and Human Communication (RO-MAN '96). 189-l 94.
• 17: James C. Lester , Sharolyn A. Converse , Susan E. Kahler , S. Todd Barlow , Brian A. Stone , Ravinder S. Bhogal, The persona effect: affective impact of animated pedagogical agents, Proceedings of the SIGCHI conference on Human factors in computing systems, p.359-366, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258797]
• 18: McNeil1, D. Hand and Mind: What Gestures Reveal About Thought. University of Chicago Press. 1992.
• 19: Scott Prevost , Peter Hodgson , Linda Cook , Elizabeth Churchill, Face-to-face interfaces, CHI '99 extended abstracts on Human factors in computing systems, May 15-20, 1999, Pittsburgh, Pennsylvania [doi>10.1145/632716.632867]
• 20: Ricke1, J., and Johnson, L. Task-Oriented Dialogs with Animated Agents in Virtual Reality. In Proceedings of the Worlcshop on Embodied Conversational Characters (Tahoe City, California, October 1998). 39-46.
• 21: Matthew David Stone, Modality in dialogue: planning, pragmatics and computation, University of Pennsylvania, Philadelphia, PA, 1998
• 22: Kristinn Runar Thorisson , Justine Cassell, Communicative humanoids: a computational model of psychosocial dialogue skills, 1996
• 23: Akikazu Takeuchi , Taketo Naito, Situated facial displays: towards social interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, p.450-455, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223965]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE

General Terms:
Experimentation, Human Factors