A biologically inspired approach to learning multimodal commands and feedback for human-robot interaction

ABSTRACT

In this paper we describe a method to enable a robot to learn how a user gives commands and feedback to it by speech, prosody and touch. We propose a biologically inspired approach based on human associative learning. In the first stage, which corresponds to the stimulus encoding in natural learning, we use unsupervised training of HMMs to model the incoming stimuli. In the second stage, the associative learning, these models are associated with a meaning using an implementation of classical conditioning. Top-down processing is applied to take into account the context as a bias for the stimulus encoding. In an experimental study we evaluated the learning of user feedback with our learning method using special training tasks, which allow the robot to explore and provoke situated feedback from the user. In this first study, the robot learned to discriminate between positive and negative feedback with an average accuracy of 95.97%.

REFERENCES

• 1: A. Austermann, S. Yamada: ""Good Robot, Bad Robot" -- Analzying User's Feedback in a Human-Robot Teaching Task", In Proc. of the RO--MAN 2008, 41--46
• 2: D. Groome: An Introduction to Cognitive Psychology. Psychology Press, Second Edition, 2008
• 3: N. Iwahashi: "Robots that learn language --Developmental Approach to Human-Machine Conversations" Proc. EELC 2006, 142--179, 2006.
• 4: T. L. Nwe, S. Foo, S. Wei; L. De Silva, "Speech emotion recognition. using hidden Markov models", Speech communication 41,4, 2003
• 5: R. Rescorla, A. Wagner: "A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement.", Classical Conditioning II, Appleton Century Crofts, 64--99, 1972
• 6: Kim, B. Scassellati, "Learning to Refine Behavior Using Prosodic Feedback", In Proc. of the ICDL 2007, pp. 205--210
• 7: The Julius Speech Recognition Project: http://julius.sourceforge.jp

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

General Terms:
Algorithms, Experimentation, Human Factors

Keywords:
human-robot-interaction, machine learning, multimodality, speech perception, user feedback