How people talk when teaching a robot

ABSTRACT

We examine affective vocalizations provided by human teachers to robotic learners. In unscripted one-on-one interactions, participants provided vocal input to a robotic dinosaur as the robot selected toy buildings to knock down. We find that (1) people vary their vocal input depending on the learner's performance history, (2) people do not wait until a robotic learner completes an action before they provide input and (3) people naively and spontaneously use intensely affective prosody. Our findings suggest modifications may be needed to traditional machine learning models to better fit observed human tendencies. Our observations of human behavior contradict the popular assumptions made by machine learning algorithms (in particular, reinforcement learning) that the reward function is stationary and path-independent for social learning interactions.

We also propose an interaction taxonomy that describes three phases of a human-teacher's vocalizations: direction, spoken before an action is taken; guidance, spoken as the learner communicates an intended action; and feedback, spoken in response to a completed action.

REFERENCES

• 1: J. Ang, R. Dhillon, A. Krupski, E. Shriberg, and A. Stolcke. Prosody-Based Automatic Detection of Annoyance and Frustration in Human-Computer Dialog. In International Conference Spoken Language Process. (ICSLP), volume 3, 2002.
• 2: C. Bartelmus. LIRC - Linux Infrared Remote Control. In http://www.lirc.org/, September 2008.
• 3: Bruce Blumberg , Marc Downie , Yuri Ivanov , Matt Berlin , Michael Patrick Johnson , Bill Tomlinson, Integrated learning for interactive synthetic characters, ACM Transactions on Graphics (TOG), v.21 n.3, July 2002
• 4: Cynthia Breazeal , Lijin Aryananda, Recognition of Affective Communicative Intent in Robot-Directed Speech, Autonomous Robots, v.12 n.1, p.83-104, January 2002 [doi>10.1023/A:1013215010749]
• 5: J. Broekens. Emotion and Reinforcement: Affective Facial Expressions Facilitate Robot Learning. Lecture Notes in Computer Science: Artificial Intelligence for Human Computing, 4451:113--132, 2007.
• 6: Carlos Busso , Zhigang Deng , Serdar Yildirim , Murtaza Bulut , Chul Min Lee , Abe Kazemzadeh , Sungbok Lee , Ulrich Neumann , Shrikanth Narayanan, Analysis of emotion recognition using facial expressions, speech and multimodal information, Proceedings of the 6th international conference on Multimodal interfaces, October 13-15, 2004, State College, PA, USA [doi>10.1145/1027933.1027968]
• 7: M. M. Chouinard and E. V. Clark. Adult Reformulations of Child Errors as Negative Evidence. Journal of Child Language, 30(03):637--669, 2003.
• 8: J. Cohen. Weighted Kappa: Nominal Scale Agreement with Provision for Scaled Disagreement or Partial Credit. Psychological Bulletin, 70(4):213--220, 1968.
• 9: Nils Dahlbäck , Arne Jönsson , Lars Ahrenberg, Wizard of Oz studies: why and how, Proceedings of the 1st international conference on Intelligent user interfaces, p.193-200, January 04-07, 1993, Orlando, Florida, United States [doi>10.1145/169891.169968]
• 10: DogsBody & Ratchet Software. MySkit - Performance Editor for PLEO. In http://www.dogsbodynet.com/myskit/index.html, January 2009.
• 11: Kevin Gold , Marek Doniec , Christopher Crick , Brian Scassellati, Robotic vocabulary building using extension inference and implicit contrast, Artificial Intelligence, v.173 n.1, p.145-166, January, 2009 [doi>10.1016/j.artint.2008.09.002]
• 12: IguanaWorks. IguanaWorks. In http://iguanaworks.net/, September 2008.
• 13: Charles Isbell , Christian R. Shelton , Michael Kearns , Satinder Singh , Peter Stone, A social reinforcement learning agent, Proceedings of the fifth international conference on Autonomous agents, p.377-384, May 2001, Montreal, Quebec, Canada [doi>10.1145/375735.376334]
• 14: L. Kaelbling, M. Littman, and A. Moore. Reinforcement Learning: A Survey. Journal of Artificial Intelligence Research, 4(2):237--285, 1996.
• 15: F. Kaplan, P.-Y. Oudeyer, E. Kubinyi, and A. Miklósi. Robotic Clicker Training. Robotics and Autonomous Systems, 38(3-4):197--206, 2002.
• 16: Ashish Kapoor , Rosalind W. Picard, Multimodal affect recognition in learning environments, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101300]
• 17: E. S. Kim and B. Scassellati. Learning to Refine Behavior Using Prosodic Feedback. In Proceedings International Conference Development and Learning (ICDL), 2007.
• 18: C. M. Lee and S. Narayanan. Toward Detecting Emotions in Spoken Dialogs. IEEE Transactions on Speech and Audio Processing, 13(2):293--303, March 2005.
• 19: D. J. Litman and K. Forbes-Riley. Recognizing Student Emotions and Attitudes on the Basis of Utterances in Spoken Tutoring Dialogues with Both Human and Computer Tutors. Speech Communication, 48(5):559--590, Feb 2006.
• 20: Clifford Nass , Jonathan Steuer , Ellen R. Tauber, Computers are social actors, Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, p.72-78, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/191666.191703]
• 21: M. Nicolescu and M. Matarić. Learning and Interacting in Human-Robot Domains. IEEE Transactions on Systems, Man and Cybernetics, 31(5):419--430, 2001.
• 22: C. Osgood, G. J. Suci, and P. H. Tannenbaum. The Measurement of Meaning. Univ. of Illinois Press, 1957.
• 23: A. Robinson-Mosher and B. Scassellati. Prosody Recognition in Male Infant-Directed Speech. In Proceedings the IEEE/RSJ International Conference Intelligent Robots and Systems (IROS), 2004.
• 24: Stuart J. Russell , Peter Norvig, Artificial Intelligence: A Modern Approach, Pearson Education, 2003
• 25: L. Steels and F. Kaplan. AIBOÜOs First Words: The Social Learning of Language and Meaning. Evolution of Communication, 4(1):3--32, 2001.
• 26: Andrew Stern , Adam Frank , Ben Resner, Virtual petz (video session): a hybrid approach to creating autonomous, lifelike dogz and catz, Proceedings of the second international conference on Autonomous agents, p.334-335, May 10-13, 1998, Minneapolis, Minnesota, United States [doi>10.1145/280765.280852]
• 27: Richard S. Sutton , Andrew G. Barto, Introduction to Reinforcement Learning, MIT Press, Cambridge, MA, 1998
• 28: A. L. Thomaz and C. Breazeal. Reinforcement Learning with Human Teachers: Evidence of Feedback and Guidance with Implications for Learning Performance. In Proceedings of the 21st National Conference on Artificial Intelligence (AAAI), 2006.
• 29: A. L. Thomaz and C. Breazeal. Transparency and Socially Guided Machine Learning. In Proceedings International Conference Development and Learning (ICDL), 2006.
• 30: UGOBE. PleoWorld - The Home of Pleo, the Robotic Baby Dinosaur from UGOBE Life Forms. In http://www.pleoworld.com, September 2008.
• 31: Zhihong Zeng , Maja Pantic , Glenn I. Roisman , Thomas S. Huang, A survey of affect recognition methods: audio, visual and spontaneous expressions, Proceedings of the 9th international conference on Multimodal interfaces, November 12-15, 2007, Nagoya, Aichi, Japan [doi>10.1145/1322192.1322216]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.9 Robotics

General Terms:
Experimentation, Human Factors

Keywords:
affective input, affective vocalization, human-robot interaction, naive teaching, reinforcement learning, social learning