Towards the design of multimodal interfaces for handheld conversational characters

ABSTRACT

This paper presents a study of individuals having conversations with animated characters on PDAs, and characterizes their use of natural nonverbal behavior compared to behavior exhibited in similar conversations with another person. The study finds that most people use the same nonverbal behavior in conversation handheld characters that they use in conversations with people, although the frequency is somewhat lower in the handheld case. These results can inform the design of new PDA input modalities which leverage the natural nonverbal behavior observed.

REFERENCES

• 1: Cassell J., McNeill, D., and McCullough, K. Speech-Gesture Mismatches: Evidence for One Underlying Representation of Linguistic and Non-Linguistic Information. Pragmatics and Cognition 6,2, 1998, 1--33.
• 2: Cassell J., Sullivan, J., Prevost, S., and Churchill, E. Embodied Conversational Agents. MIT Press, Cambridge, MA, 2000.
• 3: Cassell, J. and Thorisson, K. The Power of a Nod and a Glance: Envelope vs. Emotional Feedback in Animated Conversational Agents. Applied Artificial Intelligence 13, 1999, 519--538.
• 4: Doris M. Dehn , Susanne van Mulken, The impact of animated interface agents: a review of empirical research, International Journal of Human-Computer Studies, v.52 n.1, p.1-22, Jan. 2000 [doi>10.1006/ijhc.1999.0325]
• 5: Sharon Oviatt , Bridget Adams, Designing and evaluating conversational interfaces with animated characters, Embodied conversational agents, MIT Press, Cambridge, MA, 2001

INDEX TERMS

Keywords:
PDAs, conversational characters, embodied conversational agent, handheld computers, multimodal input