Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

Providing route directions is a complicated interaction. Utterances are combined with gestures and pronounced with appropriate timing. This study proposes a model for a robot that generates route directions by integrating three important crucial elements: utterances, gestures, and timing. Two research questions must be answered in this modeling process. First, is it useful to let robot perform gesture even though the information conveyed by the gesture is given by utterance as well? Second, is it useful to implement the timing at which humans speaks? Many previous studies about the natural behavior of computers and robots have learned from human speakers, such as gestures and speech timing. However, our approach is different from such previous studies. We emphasized the listener's perspective. Gestures were designed based on the usefulness, although we were influenced by the basic structure of human gestures. Timing was not based on how humans speak, but modeled from how they listen. The experimental result demonstrated the effectiveness of our approach, not only for task efficiency but also for perceived naturalness.

REFERENCES

• 1: Alibali, M. 2005. Gesture in Spatial Cognition: Expressing, Communicating, and Thinking About Spatial Information. Spatial Cognition and Computation. 5, 4, 307--331.
• 2: Allen, G. 2003. Gestures Accompanying Verbal Route Directions: Do They Point to a New Avenue for Examining Spatial Representations? Spatial cognition and computation. 3, 4, 259--268.
• 3: Breazeal, C., et al. 2005. Effects of nonverbal communication on efficiency and robustness in human-robot teamwork. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS2005). 383--388.
• 4: Wolfram Burgard , Armin B. Cremers , Dieter Fox , Dirk Hähnel , Gerhard Lakemeyer , Dirk Schulz , Walter Steiner , Sebastian Thrun, The interactive museum tour-guide robot, Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence, p.11-18, July 1998, Madison, Wisconsin, United States
• 5: Campione, E., et al. 2002. A Large-Scale Multilingual Study of Silent Pause Duration. Speech Prosody. 199--202.
• 6: Daniel, M., Tom, A., Manghi, E., Denis, M. 2003. Testing the Value of Route Directions through Navigational Performance. Spatial cognition and computation. 3, 4, 269--289.
• 7: Jaffe, J., and Feldstein, S. 1970. Rhythms of dialogue. Academic Press. New York.
• 8: Takayuki Kanda , Masayuki Kamasima , Michita Imai , Tetsuo Ono , Daisuke Sakamoto , Hiroshi Ishiguro , Yuichiro Anzai, A humanoid robot that pretends to listen to route guidance from a human, Autonomous Robots, v.22 n.1, p.87-100, January 2007 [doi>10.1007/s10514-006-9007-6]
• 9: Kendon, A. 2004. Gesture: Visible Action as Utterance.
• 10: Kidd, C.D., and Breazeal, C. 2004. Effect of a Robot on User Perceptions. Int. Conf. on Intelligent Robots and Systems (IROS'04). 4, 28, 3559--3564.
• 11: Kita, S. 2003. Interplay of gaze, hand, torso orientation and language in pointing. In Kita, S. (Ed.) Pointing: Where Language, Culture, and Cognition Meet.
• 12: Komori, M., Yamamoto, Y., Nagaoka, C. 2006. Manipulation of pause durations for the facilitation of understanding of speech played back at higher rate. The Japanese journal of ergonomics. 42, 2, 64--69. (in Japanese)
• 13: Kopp, S., Tepper, P.A., Ferriman, K., Striegnitz, K., and Cassell, J. 2008. Trading Spaces: How Humans and Humanoids use Speech and Gesture to Give Directions. In Nishida, T. (Ed.) Conversational Informatics: An Engineering Approach.
• 14: Krauss, R.M. 1998. Why do we gesture when we speak? Current Directions in Psychological Science. 7, 54--59.
• 15: McLaughlin, M.L. 1984. Conversation: How talk is organized.
• 16: McNeill, D. 1987. Psycholinguistics: a new approach.
• 17: McNeill, D. 2005. Gesture and thought.
• 18: Mutlu, B., Hodgins, J.K., and Forlizzi, J. 2006. A Storytelling Robot: Modeling and Evaluation of Human-like Gaze Behavior. HUMANOIDS'06. 518--523.
• 19: Nagai, Y., et al. 2003. Emergence of Joint Attention based on Visual Attention and Self Learning. Int. Symposium on Adaptive Motion of Animals and Machines.
• 20: Nagaoka, C., et al. 2005. Influence of Response Latencies on Impression Evaluation of Speakers in Dialogues. Technical Report of IEICE. 104, 745, 57--60. (in Japanese)
• 21: Ogawa H., and Watanabe, T. 2001. InterRobot: speech-driven embodied interaction robot. Advanced Robotics. 15, 3, 371--377.
• 22: Ono, T., Imai, M., Ishiguro, H. 2001. A Model of Embodied Communications with Gestures between Humans and Robots. Proc. Annual Meeting of the Cognitive Science Society (CogSci2001). 732--737.
• 23: Pacchierotti, E., Christensen, H.I., Jensfelt, P. 2006. Design of an office guide robot for social interaction studies. Int. Conf. Intelligent Robots and Systems (IROS2006), 4965--4970.
• 24: Aaron Powers , Sara Kiesler , Susan Fussell , Cristen Torrey, Comparing a computer agent with a humanoid robot, Proceedings of the ACM/IEEE international conference on Human-robot interaction, March 10-12, 2007, Arlington, Virginia, USA [doi>10.1145/1228716.1228736]
• 25: Ben Robins , Kerstin Dautenhahn , Rene te Boekhorst , Chrystopher L. Nehaniv, Behaviour delay and robot expressiveness in child-robot interactions: a user study on interaction kinesics, Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction, March 12-15, 2008, Amsterdam, The Netherlands [doi>10.1145/1349822.1349826]
• 26: Sacks, H., Schegloff, E.A., and Jefferson, G. 1974. A simplest systematic for the organization of turn-taking for conversation. Language. 50, 4, 696--735.
• 27: Scassellati, B. 2000. Investigating Models of Social Development Using a Humanoid Robot. Biorobotics. MIT Press.
• 28: Kazuhiko Shinozawa , Futoshi Naya , Junji Yamato , Kiyoshi Kogure, Differences in effect of robot and screen agent recommendations on human decision-making, International Journal of Human-Computer Studies, v.62 n.2, p.267-279, February 2005 [doi>10.1016./j.ijhcs.2004.11.003]
• 29: Toshiyuki Shiwa , Takayuki Kanda , Michita Imai , Hiroshi Ishiguro , Norihiro Hagita, How quickly should communication robots respond?, Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction, March 12-15, 2008, Amsterdam, The Netherlands [doi>10.1145/1349822.1349843]
• 30: Candace L. Sidner , Cory D. Kidd , Christopher Lee , Neal Lesh, Where to look: a study of human-robot engagement, Proceedings of the 9th international conference on Intelligent user interfaces, January 13-16, 2004, Funchal, Madeira, Portugal [doi>10.1145/964442.964458]
• 31: Candace L. Sidner , Christopher Lee , Louis-Philippe Morency , Clifton Forlines, The effect of head-nod recognition in human-robot conversation, Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction, March 02-03, 2006, Salt Lake City, Utah, USA [doi>10.1145/1121241.1121291]
• 32: Skubic, M., Blisard, S., Bailey, C., Adams, J.A., Matsakis, P. 2004. Qualitative analysis of sketched route maps: translating a sketch into linguistic descriptions. IEEE Trans. on Systems, Man, and Cybernetics. Part B, 34, 2, 1275--1282.
• 33: Striegnitz, K., Tepper, P., Lovett, A., Cassell, J. 2005. Knowledge representation for generating locating gestures in route directions. WS in Spatial Language and Dialogue.
• 34: Trafton, J.G., Cassimatis, N.L., Bugajska, M.D., Brock, D.P., Mintz, F.E., Schultz, A.C. 2005. Enabling Effective Human-Robot Interaction Using Perspective-Taking in Robots. IEEE Transactions on Systems, Man and Cybernetics. Part A, 35, 4, 460--470.
• 35: Trafton, J.G., et al. 2006. The Relationship Between Spatial Transformations and Iconic Gestures. Spatial Cognition and Computation. 6, 1, 1--29.
• 36: Yamamoto, M., and Watanabe, T. 2004. Timing control effects of utterance to communicative actions on embodied interaction with a robot. IEEE Int. Workshop on Robot and Human Communication (ROMAN2004). 467--472.
• 37: Yamamoto, M., Watanabe, T. 2006. Time Lag Effects of Utterance to Communicative Actions on CG Character-Human Greeting Interaction (ROMAN2006). 629--634.
• 38: Akiko Yamazaki , Keiichi Yamazaki , Yoshinori Kuno , Matthew Burdelski , Michie Kawashima , Hideaki Kuzuoka, Precision timing in human-robot interaction: coordination of head movement and utterance, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357077]
• 39: Brigitte Zellner, Pauses and the temporal structure of speech, Fundamentals of speech synthesis and speech recognition: basic concepts, state-of-the-art and future challenges, John Wiley and Sons Ltd., Chichester, UK, 1995
• 40: Zvonik, E., Cummins, F. 2002. Pause duration and variability in read texts. Int. Conf. on Spoken Language Processing (ICSLP-2002). 1109--1112
• 41: Zvonik, E., Cummins, F. 2003. The effect of surrounding phrase lengths on pause duration. European Conference on Speech Communication and Technology (EUROSPEECH-2003). 777--780.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

General Terms:
Design, Experimentation, Human Factors

Keywords:
gesture, route directions, timing