Visualizing non-speech sounds for the deaf

ABSTRACT

Sounds constantly occur around us, keeping us aware of our surroundings. People who are deaf have difficulty maintaining an awareness of these ambient sounds. We present an investigation of peripheral, visual displays to help people who are deaf maintain an awareness of sounds in the environment. Our contribution is twofold. First, we present a set of visual design preferences and functional requirements for peripheral visualizations of non-speech audio that will help improve future applications. Visual design preferences include ease of interpretation, glance-ability, and appropriate distractions. Functional requirements include the ability to identify what sound occurred, view a history of displayed sounds, customize the information that is shown, and determine the accuracy of displayed information. Second, we designed, implemented, and evaluated two fully functioning prototypes that embody these preferences and requirements, serving as examples for future designers and furthering progress toward understanding how to best provide peripheral audio awareness for the deaf.

REFERENCES

• 1: Auditory Visual Articulation Therapy Software, Sonido Inc., www.sonidoinc.com.
• 2: SoundForge, Hallogram Publishing, www.hallogram.com.
• 3: Alistair D. N. Edwards, Progress in Sign Languages Recognition, Proceedings of the International Gesture Workshop on Gesture and Sign Language in Human-Computer Interaction, p.13-21, September 17-19, 1997
• 4: D. Ellis, The Weft: A Representation for Periodic Sounds, Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '97)-Volume 2, p.1307, April 21-24, 1997
• 5: Elssmann, S.F. and Maki, J.E., Speech spectrographic display: use of visual feedback by hearing-impaired adults during independent articulation practice. American Annals of the Deaf, 132 (4). 276--279.
• 6: F. Wai-ling Ho-Ching , Jennifer Mankoff , James A. Landay, Can you see what i hear?: the design and evaluation of a peripheral sound display for the deaf, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642641]
• 7: Malkin, R., et al., First evaluation of acoustic event classification systems in CHIL project. In HSCMA '05 Workshop.
• 8: Oliver, N. and Horvitz, E., Layered Representations for Human Activity Recognition. In Proc. of ICMI '02, 3.
• 9: Senghas, R.J. and Monaghan, L., Signs of Their Times: Deaf Communities and the Culture of Language. Annual Review of Anthropology, 31. 69--97.
• 10: Yeung, E., et al., A wearable multichannel tactile display of voice fundamental frequency. Ear Hear, 9 (6). 342--350.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Auditory (non-speech) feedback

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: User-centered design

  K. Computing Milieux
  K.4 COMPUTERS AND SOCIETY
      K.4.2 Social Issues
          Subjects: Assistive technologies for persons with disabilities

General Terms:
Design, Human Factors

Keywords:
deaf, peripheral display, sound visualization