Can you see what i hear?: the design and evaluation of a peripheral sound display for the deaf

ABSTRACT

We developed two visual displays for providing awareness of environmental audio to deaf individuals. Based on fieldwork with deaf and hearing participants, we focused on supporting awareness of non-speech audio sounds such as ringing phones and knocking in a work environment. Unlike past work, our designs support both monitoring and notification of sounds, support discovery of new sounds, and do not require a priori knowledge of sounds to be detected. Our Spectrograph design shows pitch and amplitude, while our Positional Ripples design shows amplitude and location of sounds. A controlled experiment involving deaf participants found neither display to be significantly distracting. However, users preferred the Positional Ripples display and found that display easier to monitor (notification sounds were detected with 90% success in a laboratory setting). The Spectrograph display also supported successful detection in most cases, and was well received when deployed in the field.

REFERENCES

• 1: Arons, B. A review of the cocktail effect. Journal of the American Voice I/O Society 12 (Jul. 1992), pp. 35--50.
• 2: Asano F., et al. Sound source localization and separation in near field, IEICE Trans. Fund, E83-A(11).
• 3: Bartram, L., et al. Moving Icons: Detection and distraction. In Proceedings of Interact 2001, 2001.
• 4: Coyne K. P. and Nielsen J. How to conduct usability evaluations for accessibility: Methodology guidelines for testing websites and intranets with users who use Assistive Technology. Nielsen Norman Group, Oct, 2001, http://www.NNgroup.com/reports/accessibility/testing.
• 5: Dantzich, M., et al. Scope: Providing awareness of multiple notifications at a glance. In Proceedings of AVI 2002, 2002.
• 6: Dutta, S., et al. Evaluating benefits and distractions of animated secondary displays for attention-centric primary tasks. Regular research report in Proc. of the 2002 International Conference on Imaging Science, Systems, and Technology, 2002.
• 7: R. Elliott , J. R. W. Glauert , J. R. Kennaway , I. Marshall, The development of language processing support for the ViSiCAST project, Proceedings of the fourth international ACM conference on Assistive technologies, p.101-108, November 13-15, 2000, Arlington, Virginia, United States [doi>10.1145/354324.354349]
• 8: D. Ellis, The Weft: A Representation for Periodic Sounds, Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '97)-Volume 2, p.1307, April 21-24, 1997
• 9: Batya Friedman , David Hurley , Daniel C. Howe , Edward Felten , Helen Nissenbaum, Users' conceptions of web security: a comparative study, CHI '02 extended abstracts on Human factors in computing systems, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/506443.506577]
• 10: Blair MacIntyre , Elizabeth D. Mynatt , Stephen Voida , Klaus M. Hansen , Joe Tullio , Gregory M. Corso, Support for multitasking and background awareness using interactive peripheral displays, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502355]
• 11: Paul P. Maglio , Christopher S. Campbell, Tradeoffs in displaying peripheral information, Proceedings of the SIGCHI conference on Human factors in computing systems, p.241-248, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332438]
• 12: Lena Mamykina , Elizabeth Mynatt , Michael A. Terry, Time Aura: interfaces for pacing, Proceedings of the SIGCHI conference on Human factors in computing systems, p.144-151, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365077]
• 13: Ono Sokki Co., Sound and Sensors II. 2002. http://www.onosokki.co.jp/English/hp_e/whats_new/SV_rpt/SV_4/main.htm
• 14: Sjolander, Kare. Snack Toolkit Homepage. 2002. http://www.speech.kth.se/snack/
• 15: Thad Starner , Joshua Weaver , Alex Pentland, A Wearable Computer Based American Sign Language Recognizer, Proceedings of the 1st IEEE International Symposium on Wearable Computers, p.130, October 13-14, 1997
• 16: Robert D. Stevens , Alistair D. N. Edwards, An approach to the evaluation of assistive technology, Proceedings of the second annual ACM conference on Assistive technologies, p.64-71, April 11-12, 1996, Vancouver, British Columbia, Canada [doi>10.1145/228347.228359]
• 17: Jacob Somervell , D. Scott McCrickard , Chris North , Maulik Shukla, An evaluation of information visualization in attention-limited environments, Proceedings of the symposium on Data Visualisation 2002, May 27-29, 2002, Barcelona, Spain
• 18: US Census Bureau, Employment, Earnings and Disability - Tables. 2002. http://www.census.gov/hhes/www/disable/emperndistbl.pdf
• 19: Weiser, M. and Brown, J.S., Designing calm technology. 1995. http://www.ubiq.com/hypertext/weiser/calmtech/calmtech.htm
• 20: Craig Wisneski , Hiroshi Ishii , Andrew Dahley , Matthew G. Gorbet , Scott Brave , Brygg Ullmer , Paul Yarin, Ambient Displays: Turning Architectural Space into an Interface between People and Digital Information, Proceedings of the First International Workshop on Cooperative Buildings, Integrating Information, Organization, and Architecture, p.22-32, February 01, 1998

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.4 COMPUTERS AND SOCIETY
      K.4.2 Social Issues
          Subjects: Assistive technologies for persons with disabilities

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Audio input/output
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Auditory (non-speech) feedback; Prototyping; Evaluation/methodology; User-centered design

General Terms:
Design, Experimentation, Human Factors

Keywords:
assistive technology, deaf, non-speech audio, peripheral and ambient and notification displays, sound visualization