Nonverbal leakage in robots: communication of intentions through seemingly unintentional behavior

ABSTRACT

Human communication involves a number of nonverbal cues that are seemingly unintentional, unconscious, and automatic-both in their production and perception-and convey rich information on the emotional state and intentions of an individual. One family of such cues is called "nonverbal leakage." In this paper, we explore whether people can read nonverbal leakage cues-particularly gaze cues-in humanlike robots and make inferences on robots' intentions, and whether the physical design of the robot affects these inferences. We designed a gaze cue for Geminoid-a highly humanlike android-and Robovie-a robot with stylized, abstract humanlike features-that allowed the robots to "leak" information on what they might have in mind. In a controlled laboratory experiment, we asked participants to play a game of guessing with either of the robots and evaluated how the gaze cue affected participants' task performance. We found that the gaze cue did, in fact, lead to better performance, from which we infer that the cue led to attributions of mental states and intentionality. Our results have implications for robot design, particularly for designing expression of intentionality, and for our understanding of how people respond to human social cues when they are enacted by robots.

REFERENCES

• 1: Ambady, N. and Rosenthal, R. Thin Slides of Expressive Behavior as Predictors of Interpersonal Consequences: A Meta-Analysis. Psychological Bulletin, 111 (2), 256--274, 1992.
• 2: Argyle, M. et al. The communication of friendly and hostile attitudes by verbal and non-verbal signals. European J. Social Psychology, 1 (3), 385--402, 1971.
• 3: Jeremy N. Bailenson , Nick Yee , Kayur Patel , Andrew C. Beall, Detecting digital chameleons, Computers in Human Behavior, v.24 n.1, p.66-87, January, 2008 [doi>10.1016/j.chb.2007.01.015]
• 4: Baron-Cohen, S. Mindblindness: An Essay on Autism and Theory of Mind. MIT Press, 1995.
• 5: Bartneck, C. et al. In your face, robot! The influence of a character's embodiment on how users perceive its emotional expressions. In Proc. D&E'04, 2004.
• 6: Bayliss, A. P. and Tipper, S. P. Predictive Gaze Cues and Personality Judgments: Should Eye Trust You? Psych. Science, 17 (6), 514--520, 2006.
• 7: Blakemore, S. J. and Decety, J. From the perception of action to the understanding of intention. Nature Reviews Neuroscience, 2 (8), 561--567, 2001.
• 8: Breazeal, C. and Scassellati, B. How to build robots that make friends and influence people. In Proc. IROS, 1999.
• 9: Breazeal, C. Emotive qualities in robot speech. In Proc. IROS'01, 2001.
• 10: Cynthia Breazeal, Designing Sociable Robots, MIT Press, Cambridge, MA, 2002
• 11: Bruce, A. et al. The role of expressiveness and attention in human-robot interaction. In Proc. ICRA'02, 4138--4142, 2002.
• 12: Buller, D. B. et al. Testing interpersonal deception theory: The language of interpersonal deception. Comm. Theory, 6, 268--289, 1996.
• 13: Calder, A. J. et al. Reading the mind from eye gaze. Neuropsychologia, 40, 1129--1138, 2002.
• 14: Castelli, F. et al. Autism, Asperger syndrome and brain mechanisms for the attribution of mental states to animated shapes. Brain, 125, 1839--1849, 2002.
• 15: Castiello, U. Understanding other people's actions: Intention and attention. J. Exp. Psych.: Human Perc. & Perf., 29, 416--430, 2003.
• 16: Chartrand, T. L. and Bargh, J. A. The chameleon effect: the perception-behavior link and social interaction. J. Pers. & Soc. Psych., 76, 893--910, 1999.
• 17: Chawla, P. and Krauss, R. M. Gesture and speech in spontaneous and rehearsed narratives. J. Exp. Soc. Psych., 30 (6), 580--601, 1994.
• 18: DePaulo, B. M. et al. Cues to Deception. Psych. Bulletin. 129 (1), 74--118, 2003.
• 19: Ekman, P. and Friesen, W.V. Nonverbal leakage and clues to deception. Psychiatry, 32, 88--106, 1969.
• 20: Ekman, P. et al. The Duchenne smile: Emotional expression and brain physiology II. J. Pers. & Soc. Psych., 58, 342--353, 1990.
• 21: Emery, N. J. The eyes have it: the neuroethology, function and evolution of social gaze. Neurosc. & Biobehav. Rev., 24 (6), 581--604, 2000.
• 22: Feldman, R. S. et al. Nonverbal Cues as Indicators of Verbal Dissembling. American Ed. Res. J., 15 (2), 217--231, 1978.
• 23: Freire, A. et al. Are Eyes Windows to a Deceiver's Soul? Children's Use of Another's Eye Gaze Cues in a Deceptive Situation. Dev. Psych., 40 (6), 1093--1104, 2004.
• 24: Frischen, A. et al. Gaze Cueing of Attention: Visual Attention, Social Cognition, and Individual Differences. Psych. Bulletin, 133 (4), 694--724, 2007.
• 25: Hemsley, G. D. and Doob, A. N. The effect of looking behavior on perceptions of a communicator's credibility. J. Appl. Soc. Psych., 8, 136--144, 1978.
• 26: Pamela J. Hinds , Teresa L. Roberts , Hank Jones, Whose job is it anyway? a study of human-robot interaction in a collaborative task, Human-Computer Interaction, v.19 n.1, p.151-181, June 2004 [doi>10.1207/s15327051hci1901&2_7]
• 27: Imai, M. et al. Robot mediated round table: Analysis of the effect of robot's gaze. In Proc. ROMAN'02, 411--416, 2002.
• 28: Imai, M. et al. Physical relation and expression: joint attention for human-robot interaction. IEEE Trans. Ind. Elec., 50 (4), 636--643, 2003.
• 29: Rachel Gockley , Jodi Forlizzi , Reid Simmons, Interactions with a moody robot, Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction, March 02-03, 2006, Salt Lake City, Utah, USA [doi>10.1145/1121241.1121274]
• 30: Goetz, J. et al. Matching robot appearance and behavior to tasks to improve human-robot cooperation. In Proc. ROMAN'03, 2003.
• 31: Goffman, E. Relations in public. New York: Harper & Row, 1971.
• 32: Goldberg, L. R. et al. The International Personality Item Pool and the future of public-domain personality measures. Journal of Research in Personality, 40, 84--96, 2006.
• 33: Kanda, T. et al. Body Movement Analysis of Human-Robot Interaction. In Proc. IJCAI'03, 177--182, 2003.
• 34: Sara Kiesler , Jennifer Goetz, Mental models of robotic assistants, CHI '02 extended abstracts on Human factors in computing systems, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/506443.506491]
• 35: Kleinke, C. L. Gaze and eye contact: A research review. Psychological Bulletin, 10, 78--100, 1986.
• 36: Krauss, R. M. et al. Verbal, vocal and visible factors in judgments of another's affect. J. Pers. & Soc. Psych., 40, 312--320, 1981.
• 37: Kraut, R. E. and Poe, D. Behavioral roots of person perception: The deception judgments of customs inspectors and laymen. J. Pers. & Soc. Psych., 39, 784--798, 1980.
• 38: Leslie, A. M. Pretense and representation: The origins of "theory of mind". Psych. Review, 94, 412--426, 1987.
• 39: Miklósi, A. et al. Intentional behavior in dog-human communication: An experimental analysis of "showing" behavior in the dog. Animal Cognition, 3 (3), 159--166, 2000.
• 40: Miwa, H. et al. Effective Emotional Expressions with Emotion Expression Humanoid Robot WE-4RII. In Proc. IROS'04, 2004.
• 41: Mutlu, B. et al. A Storytelling Robot: Modeling and Evaluation of Human-like Gaze Behavior. In Proc. Humanoids, 2006.
• 42: Nowak, K. Defining and differentiating copresence, social presence and presence as transportation. In Proc. Presence, 2001.
• 43: Kristine L. Nowak , Frank Biocca, The effect of the agency and anthropomorphism of users' sense of telepresence, copresence, and social presence in virtual environments, Presence: Teleoperators and Virtual Environments, v.12 n.5, p.481-494, October 2003 [doi>10.1162/105474603322761289]
• 44: Parise, S. et al. Cooperating with life-like interface agents. IBM WRC Technical Report, 1998.
• 45: Perrett, D. I. and Emery, N. J. Understanding the intentions of others from visual signals: neurophysiological evidence. Cahiers de Psychologie Cognitive, 13, 683--694, 1994.
• 46: Andrea C. Pierno , Cristina Becchio , Matthew B. Wall , Andrew T. Smith , Luca Turella , Umberto Castiello, When Gaze Turns into Grasp, Journal of Cognitive Neuroscience, v.18 n.12, p.2130-2137, December 2006 [doi>10.1162/jocn.2006.18.12.2130]
• 47: Scheeff, M. et al. Experiences with Sparky: A social robot. In Proc. Workshop on Interactive Robot Entertainment, 2000.
• 48: Scherer, K. R. et al. Minimal cues in the vocal communication of affect: Judging emotion from content-masked speech. J. Psycholinguistic Research, 1, 269--285, 1972.
• 49: Scherer, K. R. et al. The voice of confidence: Paralinguistic cues and audience evaluation. J. Res. in Pers., 7, 31--44, 1973.
• 50: Candace L. Sidner , Cory D. Kidd , Christopher Lee , Neal Lesh, Where to look: a study of human-robot engagement, Proceedings of the 9th international conference on Intelligent user interfaces, January 13-16, 2004, Funchal, Madeira, Portugal [doi>10.1145/964442.964458]
• 51: Sugiyama, O. et al. Three-layered draw-attention model for humanoid robots with gestures and verbal cues. In Proc. IROS'05, 2005.
• 52: Surakka, V. and Hietanen, J. K. Facial and emotional reactions to Duchenne and non-Duchenne smiles. Intl. J. Psychophysiology, 29, 23--33, 1998.
• 53: Tojo, T. et al. A conversational robot utilizing facial and body expressions. In Proc. Systems, Man, and Cybernetics'00, 2000.
• 54: J. Gregory Trafton , Magda D. Bugajska , Benjamin R. Fransen , Raj M. Ratwani, Integrating vision and audition within a cognitive architecture to track conversations, Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction, March 12-15, 2008, Amsterdam, The Netherlands [doi>10.1145/1349822.1349849]
• 55: Watson D. et al. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers. Soc. Psychol., 54 (6), 1063--1070, 1988.
• 56: Waxer, P. H. Nonverbal cues for anxiety: an examination of emotional leakage. J. Abnormal Psych., 86 (3), 306--314, 1977.
• 57: Williams, L. M. et al. In Search of the "Duchenne Smile": Evidence from Eye Movements. J. Psychophysiology, 15 (2), 122--127, 2001.
• 58: Zuckerman, M. et al. Verbal and nonverbal communication of deception. In L. Berkowitz (Ed.), Advances in experimental social psychology, 14, 1--59. New York: Academic Press, 1981.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: User-centered design; Evaluation/methodology

General Terms:
Design, Human Factors

Keywords:
gaze, geminoid, humanlikeness, nonverbal behavior, nonverbal leakage, robovie