Voicedraw: a hands-free voice-driven drawing application for people with motor impairments

ABSTRACT

We present VoiceDraw, a voice-driven drawing application for people with motor impairments that provides a way to generate free-form drawings without needing manual interaction. VoiceDraw was designed and built to investigate the potential of the human voice as a modality to bring fluid, continuous direct manipulation interaction to users who lack the use of their hands. VoiceDraw also allows us to study the issues surrounding the design of a user interface optimized for non-speech voice-based interaction. We describe the features of the VoiceDraw application, our design process, including our user-centered design sessions with a 'voice painter', and offer lessons learned that could inform future voice-based design efforts. In particular, we offer insights for mapping human voice to continuous control.

REFERENCES

• 1: AMFPA - Association of Mouth and Foot Painting Artists of the World. http://www.amfpa.com/.
• 2: Eric Bergman , Earl Johnson, Toward accessible human-computer interaction, Advances in human-computer interaction (vol. 5), Ablex Publishing Corp., Norwood, NJ, 1995
• 3: Jeff A. Bilmes , Xiao Li , Jonathan Malkin , Kelley Kilanski , Richard Wright , Katrin Kirchhoff , Amarnag Subramanya , Susumu Harada , James A. Landay , Patricia Dowden , Howard Chizeck, The vocal joystick: a voice-based human-computer interface for individuals with motor impairments, Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, p.995-1002, October 06-08, 2005, Vancouver, British Columbia, Canada [doi>10.3115/1220575.1220700]
• 4: Dai, L., Goldman, R., Sears, A. and Lozier, J. (2004) Speech-based cursor control: a study of grid-based solutions. Proc. ASSETS '04. New York: ACM Press, 94--101.
• 5: Melissa Dawe, Desperately seeking simplicity: how young adults with cognitive disabilities and their families adopt assistive technologies, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124943]
• 6: de Mauro, C., Gori, M., Maggini, M. and Martinelli, E. (2001) Easy access to graphical interfaces by voice mouse. Technical report, Università di Siena. Available from the author at: [e-mail].
• 7: Fuller, R. B. (1973) Nine Chains to the Moon. Cape.
• 8: François Guimbretiére , Terry Winograd, FlowMenu: combining command, text, and data entry, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.213-216, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354778]
• 9: Susumu Harada , James A. Landay , Jonathan Malkin , Xiao Li , Jeff A. Bilmes, The vocal joystick:: evaluation of voice-based cursor control techniques, Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility, October 23-25, 2006, Portland, Oregon, USA [doi>10.1145/1168987.1169021]
• 10: Anthony J. Hornof , Anna Cavender, EyeDraw: enabling children with severe motor impairments to draw with their eyes, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1054995]
• 11: Takeo Igarashi , John F. Hughes, Voice as sound: using non-verbal voice input for interactive control, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502372]
• 12: Simeon Keates , Faustina Hwang , Patrick Langdon , P. John Clarkson , Peter Robinson, Cursor measures for motion-impaired computer users, Proceedings of the fifth international ACM conference on Assistive technologies, July 08-10, 2002, Edinburgh, Scotland [doi>10.1145/638249.638274]
• 13: Rick Kjeldsen, Improvements in vision-based pointer control, Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility, October 23-25, 2006, Portland, Oregon, USA [doi>10.1145/1168987.1169020]
• 14: Golan Levin , Zachary Lieberman, In-situ speech visualization in real-time interactive installation and performance, Proceedings of the 3rd international symposium on Non-photorealistic animation and rendering, June 07-09, 2004, Annecy, France [doi>10.1145/987657.987659]
• 15: Rainer Malkewitz, Head pointing and speech control as a hands-free interface to desktop computing, Proceedings of the third international ACM conference on Assistive technologies, p.182-188, April 15-17, 1998, Marina del Rey, California, United States [doi>10.1145/274497.274531]
• 16: Bill Manaris , Renée McCauley , Valanne MacGyvers, An Intelligent Interface for Keyboard and Mouse Control -- Providing Full Access to PC Functionality via Speech, Proceedings of the Fourteenth International Florida Artificial Intelligence Research Society Conference, p.182-188, May 21-23, 2001
• 17: Yoshiyuki Mihara , Etsuya Shibayama , Shin Takahashi, The migratory cursor: accurate speech-based cursor movement by moving multiple ghost cursors using non-verbal vocalizations, Proceedings of the 7th international ACM SIGACCESS conference on Computers and accessibility, October 09-12, 2005, Baltimore, MD, USA [doi>10.1145/1090785.1090801]
• 18: Phillips, B. and Zhao, H. (1993) Predictors of assistive technology abandonment. Assistive Technology, 5(1):36--45.
• 19: J. Sporka , H. Kurniawan , Pavel Slavík, Acoustic control of mouse pointer, Universal Access in the Information Society, v.4 n.3, p.237-245, February 2006 [doi>10.1007/s10209-005-0010-z]
• 20: Mark A. Tapia , Gordon Kurtenbach, Some design refinements and principles on the appearance and behavior of marking menus, Proceedings of the 8th annual ACM symposium on User interface and software technology, p.189-195, November 15-17, 1995, Pittsburgh, Pennsylvania, United States [doi>10.1145/215585.215973]
• 21: Voice painter. http://www.voicepainter.com/index.php.
• 22: Warren, B. (1997) Ch. 13 in Quality of Life for People with Disabilities (2nd ed.). Nelson Thornes, 270--291.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Voice I/O

General Terms:
Design, Human Factors

Keywords:
computer art, continuous input, drawing, motor impairments, painting, speech recognition, voice-based user interfaces