The VoiceBot: a voice controlled robot arm

ABSTRACT

We present a system whereby the human voice may specify continuous control signals to manipulate a simulated 2D robotic arm and a real 3D robotic arm. Our goal is to move towards making accessible the manipulation of everyday objects to individuals with motor impairments. Using our system, we performed several studies using control style variants for both the 2D and 3D arms. Results show that it is indeed possible for a user to learn to effectively manipulate real-world objects with a robotic arm using only non-verbal voice as a control mechanism. Our results provide strong evidence that the further development of non-verbal voice controlled robotics and prosthetic limbs will be successful.

REFERENCES

• 1: Jeff A. Bilmes , Xiao Li , Jonathan Malkin , Kelley Kilanski , Richard Wright , Katrin Kirchhoff , Amarnag Subramanya , Susumu Harada , James A. Landay , Patricia Dowden , Howard Chizeck, The vocal joystick: a voice-based human-computer interface for individuals with motor impairments, Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, p.995-1002, October 06-08, 2005, Vancouver, British Columbia, Canada [doi>10.3115/1220575.1220700]
• 2: Young Sang Choi , Cressel D. Anderson , Jonathan D. Glass , Charles C. Kemp, Laser pointers and a touch screen: intuitive interfaces for autonomous mobile manipulation for the motor impaired, Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility, October 13-15, 2008, Halifax, Nova Scotia, Canada [doi>10.1145/1414471.1414512]
• 3: Susumu Harada , James A. Landay , Jonathan Malkin , Xiao Li , Jeff A. Bilmes, The vocal joystick:: evaluation of voice-based cursor control techniques, Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility, October 23-25, 2006, Portland, Oregon, USA [doi>10.1145/1168987.1169021]
• 4: Susumu Harada , Jacob O. Wobbrock , James A. Landay, Voicedraw: a hands-free voice-driven drawing application for people with motor impairments, Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility, October 15-17, 2007, Tempe, Arizona, USA [doi>10.1145/1296843.1296850]
• 5: Susumu Harada , Jacob O. Wobbrock , Jonathan Malkin , Jeff A. Bilmes , James A. Landay, Longitudinal study of people learning to use continuous voice-based cursor control, Proceedings of the 27th international conference on Human factors in computing systems, April 04-09, 2009, Boston, MA, USA [doi>10.1145/1518701.1518757]
• 6: Takeo Igarashi , John F. Hughes, Voice as sound: using non-verbal voice input for interactive control, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502372]
• 7: KIM, S.-P., SIMERAL, J. D., HOCHBERG, L. R., DONOGHUE, J. P., AND BLACK, M. J. Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia. Journal of Neural Engineering 5, 4 (2008), 455--476.
• 8: LI, X., MALKIN, J., BILMES, J., HARADA, S., AND LANDAY, J. An online adaptive filtering algorithm for the Vocal Joystick. In Interspeech (Pittsburgh, PA, Sept. 2006).
• 9: LI, Y., WANG, C., ZHANG, H., AND GUAN, C. An eeg-based bci system for 2d cursor control. IEEE Int'l Joint Conf. on Neural Networks (June 2008), 2214--2219.
• 10: MALKIN, J., LI, X., AND BILMES, J. Energy and loudness for speed control in the Vocal Joystick. In Automatic Speech Recognition and Understanding (Dec. 2005), pp. 409--414.
• 11: MARTENS, C., RUCHEL, N., LANG, O., AND GRASER, A. A FRIEND for assisting handicapped people. IEEE Robotics and Automation Magazine 8, 1 (2001), 57--65.
• 12: Yoshiyuki Mihara , Etsuya Shibayama , Shin Takahashi, The migratory cursor: accurate speech-based cursor movement by moving multiple ghost cursors using non-verbal vocalizations, Proceedings of the 7th international ACM SIGACCESS conference on Computers and accessibility, October 09-12, 2005, Baltimore, MD, USA [doi>10.1145/1090785.1090801]
• 13: REICHENSPURNER, H., ET AL. Use of the voice-controlled and computer-assisted surgical system ZEUS for endoscopic coronary artery bypass grafting. J. Thoracic and Cardiovascular Surgery 118 (1999), 11--16.
• 14: SHENOY, P., KRAULEDAT, M., BLANKHERTZ, B., RAO, R., AND MUELLER, K.-R. Towards adaptive classification for BCI. J. Neural Eng. 3 (2006), R13--R23.
• 15: Adam J. Sporka , Pavel Slavík, Vocal control of a radio-controlled car, ACM SIGACCESS Accessibility and Computing [doi>10.1145/1394427.1394428]
• 16: STEVENS, S. S., VOLKMANN, J., AND NEWMAN, E. B. A scale for the measurement of the psychological magnitude pitch. J. of Acoustical Society of America 8, 3 (1937), 185--190.
• 17: WANG, L.-C., AND CHEN, C. A combined optimization method for solving the inverse kinematics problem of mechanical manipulators. IEEE Trans. on Robotics and Automation 7, 4 (1991), 489--499.
• 18: WELMAN, C. Inverse kinematics and geometric constraints for articulated figure manipulation. Master's thesis, Simon Fraser University, Burnaby, BC, 1993.
• 19: Sook Young Won , Dong-In Lee , Julius Smith, Humming control interface for hand-held devices, Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility, October 15-17, 2007, Tempe, Arizona, USA [doi>10.1145/1296843.1296901]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Voice I/O

Additional Classification:
  K. Computing Milieux
  K.4 COMPUTERS AND SOCIETY
      K.4.2 Social Issues
          Subjects: Assistive technologies for persons with disabilities

General Terms:
Experimentation

Keywords:
motor impairment, robotics, speech recognition, voice-based interface