Auditory and tactile interfaces for representing the visual effects on the web

ABSTRACT

In this paper, we describe auditory and tactile interfaces to represent visual effects nonvisually for blind users, allowing intuitive recognition of visual content that appears on the Web. This research examines how visual effects could be recognized by blind subjects using the senses of hearing and touch, aiming at integrating the results into a practical system in the future. As an initial step, two experiments were performed, one for sonification and tactilization of a page overview based on color-based fragmented groupings without speech, and one for sonification and tactilization of emphasized text based on analyzing rich text information with speech. The subjects could recognize visual representations presented by auditory and tactile interfaces throughout the experiment, and were conscious of the importance of the visual structures. We believe this shows our approach may be practical and available in the future.We will summarize our results and discuss what kind of information is suitable for each sense, as well as the next planned experiment and other future work.

REFERENCES

• 1: Chieko Asakawa , Takashi Itoh, User interface of a Home Page Reader, Proceedings of the third international ACM conference on Assistive technologies, p.149-156, April 15-17, 1998, Marina del Rey, California, United States [doi>10.1145/274497.274526]
• 2: IBM HomePage Reader, Inc. IBM, http://www-3.ibm.com/able/hprdoc.html
• 3: Freedom Scientific, Inc. Jaws. http://www.freedomsci.com/.
• 4: GW Micro, Inc. Window-Eyes. http://www.gwmicro.com/.
• 5: Frankie James, AHA: audio HTML access, Selected papers from the sixth international conference on World Wide Web, p.1395-1404, September 1997, Santa Clara, California, United States
• 6: Patrick Roth , Lori Petrucci , Thierry Pun , André Assimacopoulos, Auditory browser for blind and visually impaired users, CHI '99 extended abstracts on Human factors in computing systems, May 15-20, 1999, Pittsburgh, Pennsylvania [doi>10.1145/632716.632852]
• 7: Elizabeth D. Mynatt , Gerhard Weber, Nonvisual presentation of graphical user interfaces: contrasting two approaches, Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, p.166-172, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/191666.191732]
• 8: Stephen A. Brewster, Using nonspeech sounds to provide navigation cues, ACM Transactions on Computer-Human Interaction (TOCHI), v.5 n.3, p.224-259, Sept. 1998 [doi>10.1145/292834.292839]
• 9: Elizabeth D. Mynatt, Designing with auditory icons: how well do we identify auditory cues?, Conference companion on Human factors in computing systems, p.269-270, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/259963.260483]
• 10: M. Blattner, D.A. Sumikawa and R.M. Greenberg, "Earcons and icons: Their structure and common design principles," Human-Computer Interaction, Vol.4, No.l, pp.11--44, 1989.
• 11: Chieko Asakawa , Hironobu Takagi, Annotation-based transcoding for nonvisual web access, Proceedings of the fourth international ACM conference on Assistive technologies, p.172-179, November 13-15, 2000, Arlington, Virginia, United States [doi>10.1145/354324.354588]
• 12: Schmidt, F. (eds.). Fundamentals of Sensory Physiology. Springer, New York, 1979.
• 13: Ohyama, T., Imai, S. (eds.). Handbook of Sensory and Cognitive Psychology. Seishinshobo, Tokyo, 1994.
• 14: Ifukube, T. Sound-based Assisitive Technology. Corona Publishing Co, 1997.
• 15: Ifukube, T. A Study of HCI for People in Japan with Communication Disorders, in Proceedings UAHCI2001 (August 2001), Lawrence Erlbaum, 924--928.

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.4 COMPUTERS AND SOCIETY
      K.4.2 Social Issues
          Subjects: Assistive technologies for persons with disabilities

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Auditory (non-speech) feedback; Input devices and strategies (e.g., mouse, touchscreen)

General Terms:
Algorithms, Documentation, Experimentation, Human Factors

Keywords:
auditory interface, blind, nonvisual, sonification, tactile interface, tactilization