Striking a c[h]ord: vocal interaction in assistive technologies, games, and more

ABSTRACT

Vocal interaction research has primarily been focused on the use of systems for automatic speech recognition and synthesis. Whilst ASR has been successful in various domains, it can be impractical in some contexts of use such as in time-sensitive and continuous controls and in applications involving users with speech impairment. This workshop aims to discuss the state of the art in vocal interaction methods that go beyond word recognition by exploiting the information contained within non-verbal vocalizations (e.g. pitch, volume, or timbre). The overarching objective of this workshop is to sketch a research agenda on the topic of the emerging discipline of non-verbal vocal interaction and its implications for the design of interactive systems. The workshop will be of interest to researchers, designers, developers, and users that are interested or would benefit from use of the non-verbal interaction.

REFERENCES

• 1: Al-Hashimi, S. Blowtter: A Voice-Controlled Plotter. To appear in Proc. HCI 2006 Engage.
• 2: Bilmes, J., Malkin, J., Li, X., Harada, S., Kilanski, K., Kirchhoff, K., Wright, R., Subramanya, A., Landay, J., Dowden, P., Chizeck, H.: The Vocal Joystick. In Proc. IEEE Intl Conf Audio, Speech and Sig Proc. Toulouse, France, IEEE (2006).
• 3: Meghan Deutscher , Reynald Hoskinson , Sachiyo Takashashi , Sidney Fels, Echology: an interactive spatial sound and video artwork, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101347]
• 4: Goto, M., Itou, K., Akiba, T., Hayamizu, S.: Speech Completion: New Speech Interface with On-demand Completion Assistance. In Proc. HCI International 2001, Vol.1, pp. 198--202, August 2001.
• 5: Goto, M., Itou, K., Kobayashi, T. Speech interface exploiting intentionally--controlled nonverbal speech information. In Proc. UIST 2005, ACM Press (2005), 35--36.
• 6: Hamalainen, P., Maki-Patola, T., Pulkki, V., Airas, M. Musical computer games played by singing. In Proc 7th Intl Conf on Digital Audio Effects, Naples, Italy, 2004, 367--371.
• 7: Takeo Igarashi , John F. Hughes, Voice as sound: using non-verbal voice input for interactive control, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502372]
• 8: Jacob, R. J. K., Leggett, J. J., Myers, B. A. and Pausch, R. Interaction styles and input/output devices. Behaviour & Information Technology 12, 2 (1993), 69--79.
• 9: Nakano, T., Goto, M., Ogata, J., Hiraga, Y. Voice drummer: a music notation interface of drum sounds using voice percussion input. In Proc. UIST 2000, ACM Press (2005), 49--50.
• 10: Paul P. Maglio , Teenie Matlock , Christopher S. Campbell , Shumin Zhai , Barton A. Smith, Gaze and Speech in Attentive User Interfaces, Proceedings of the Third International Conference on Advances in Multimodal Interfaces, p.1-7, October 14-16, 2000
• 11: Sporka, A. J., Kurniawan, S. H., Slavik, P. Whistling user interface (U3I). In Proc. UI4All 2004, LNCS 3196, Springer-Verlag (2004), 472--478.
• 12: Sporka, A. J., Kurniawan, S. H., Slavik, P.: Non-speech Operated Emulation of Keyboard. In Clarkson J., Langdon, P., Robinson P. (eds) Designing Accessible Technology, Springer--Verlag (2006), 145--154.
• 13: Vilimek, R., Hempel, T. Effects of Speech and Non-Speech Sounds on Short-Term Memory and Possible Implications for In-Vehicle Use. In Proc. ICAD'05.
• 14: Welch, G. F., Howard, D. M., Rush, C. Real-time visual feedback in the development of vocal pitch accuracy in singing. Psychology of Music, 17(2), 146--157.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Software psychology
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Theory and methods

General Terms:
Design, Experimentation, Human Factors, Measurement

Keywords:
acoustic interaction, assistive technology, emerging interaction styles, entertainment, human voice, non-verbal input