Experiments in socially guided machine learning: understanding how humans teach

ABSTRACT

In Socially Guided Machine Learning we explore the ways in which machine learning can more fully take advantage of natural human interaction. In this work we are studying the role real-time human interaction plays in training assistive robots to perform new tasks. We describe an experimental platform, Sophie's World, and present descriptive analysis of human teaching behavior found in a user study. We report three important observations of how people administer reward and punishment to teach a simulated robot a new task through Reinforcement Learning. People adjust their behavior as they develop a model of the learner, they use the reward channel for guidance as well as feedback, and they may also use it as a motivational channel.

REFERENCES

• 1: Charles Isbell , Christian R. Shelton , Michael Kearns , Satinder Singh , Peter Stone, A social reinforcement learning agent, Proceedings of the fifth international conference on Autonomous agents, p.377-384, May 2001, Montreal, Quebec, Canada [doi>10.1145/375735.376334]
• 2: L. P. Kaelbling, M. L. Littman, and A. P. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237--285, 1996.
• 3: Maja J. Matarić, Reinforcement Learning in the Multi-Robot Domain, Autonomous Robots, v.4 n.1, p.73-83, March 1997 [doi>10.1023/A:1008819414322]
• 4: S. Thrun. Robotics. In S. Russell and P. Norvig, editors, Artificial Intelligence: A Modern Approach (2nd edition). Prentice Hall, 2002.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE

General Terms:
Algorithms, Human Factors

Keywords:
human-robot interaction, machine learning, socially guided agents