A computer-animated tutor for spoken and written language learning

ABSTRACT

Baldi, a computer-animated talking head is introduced. The quality of his visible speech has been repeatedly modified and evaluated to accurately simulate naturally talking humans. Baldi's visible speech can be appropriately aligned with either synthesized or natural auditory speech. Baldi has had great success in teaching vocabulary and grammar to children with language challenges and training speech distinctions to children with hearing loss and to adults learning a new language. We demonstrate these learning programs and also demonstrate several other potential application areas for Baldi®.

REFERENCES

• 1: Barker, L. J. (in press). Computer-assisted vocabulary acquisition: The CSLU vocabulary tutor in oral-deaf education. Journal of Deaf Studies and Deaf Education, in press.
• 2: Bosseler, A. and Massaro, D. W. (in press). Development and Evaluation of a Computer-Animated Tutor for Vocabulary and Language Learning for Children with Autism. Journal of Autism and Developmental Disorders, in press. http://mambo.ucsc.edu/pdf/autism.pdf
• 3: Cohen, M. M., Beskow, J., & Massaro, D. W. (1998). Recent developments in facial animation: An inside view. In D. Burnham, J. Robert-Ribes, & E. Vatikiotis-Bateson (Eds.) Proceedings of Auditory Visual Speech Perception '98. (pp. 201--206). Terrigal-Sydney Australia, December, 1998. AVSP '98 (December 4-6, 1998, Sydney, Australia).
• 4: Massaro, D. W. (1998). Perceiving Talking Faces: From Speech Perception to a Behavioral Principle. MIT Press: Cambridge, MA.
• 5: Massaro, D. W., & Light, J. (in press). Improving the vocabulary of children with hearing loss, Volta Review, in press.
• 6: Massaro, D. W., & Light, J. (2003). Read My Tongue Movements: Bimodal Learning To Perceive And Produce Non-Native Speech /r/ and /l/. Eurospeech 2003-Switzerland (Interspeech). 8th European Conference on Speech Communication and Technology, Geneva, Switzerland.
• 7: Massaro, D. W., & Light, J. (in press). Using Visible Speech for Training Perception and Production of Speech for Hard of Hearing Individuals. Volta Review, in press.
• 8: Vygotsky, L. (1962). Thought and language. Cambridge, MA: MIT Press.

INDEX TERMS

Primary Classification:
  J. Computer Applications
  J.4 SOCIAL AND BEHAVIORAL SCIENCES
      Subjects: Psychology

General Terms:
Experimentation, Human Factors, Languages, Performance

Keywords:
facial and speech synthesis, language learning