Mapping GUIs to auditory interfaces

ABSTRACT

This paper describes work to provide mappings between X-based graphical interfaces and auditory interfaces. In our system, dubbed Mercator, this mapping is transparent to applications. The primary motivation for this work is to provide accessibility to graphical applications for users who are blind or visually impaired. We describe the design of an auditory interface which simulates many of the features of graphical interfaces. We then describe the architecture we have built to model and transform graphical interfaces. Finally, we conclude with some indications of future research for improving our translation mechanisms and for creating an auditory “desktop” environment.

REFERENCES

• BBV90: L.H. Boyd, W.L. Boyd, and G.C. Vanderheiden. The graphical user interface: Crisis, danger and opportunity. Journal of Visual Impairment and Blindness, pages 496-502, December 1990.
• BGB91: Bill Buxton, Bill Gaver, and Sara Bly. The Use of Non-SpeechAudio at the Interface. Tutorial Presented at CHI'91. April 1991.
• Bur92: David Burgess. Low Cost Sound Spatilization. To appear in UIST '92: The Fifth Annual Symposium on User Interface Software and Technology and Technolgy, November 1992.
• Bux86: W. Buxton , R. Foulds , M. Rosen , L. Scadden , F. Shein, Human interface design and the handicapped user, Proceedings of the SIGCHI conference on Human factors in computing systems, p.291-297, April 13-17, 1986, Boston, Massachusetts, United States
• Che53: E.C. Cherry. Some experiments on the recognition of speech with one and two ears. Journal of the Acoustical Society of America, 22, pages 61-62.
• HC86: D. Austin Henderson, Jr. , Stuart Card, Rooms: the use of multiple virtual workspaces to reduce space contention in a window-based graphical user interface, ACM Transactions on Graphics (TOG), v.5 n.3, p.211-243, July 1986 [doi>10.1145/24054.24056]
• Edw89: Alistair D. N. Edwards, Modelling blind users' interactions with an auditory computer interface, International Journal of Man-Machine Studies, v.30 n.5, p.575-589, May 1989
• Gav89: William W. Gaver. The sonicfinder: An interface that uses auditory icons. Human Computer Interaction, 4:67-94, 1989.
• HTAP90: HumanWare, Artic Technologies, ADHOC, and The Reader Project. Making good decisions on technology: Access solutions for blindness and low vision. In Closing the Gap Conference, October 1990. Industry Experts Panel Discussion.
• Lad88: R. E. Ladner , F. A. McDonough , W. Roth , L. A. Scadden , G. C. Vanderheiden, Public Law 99-506, “Section 508” Electronic Equipment Accessibility for disabled workers, Proceedings of the SIGCHI conference on Human factors in computing systems, p.219-222, May 15-19, 1988, Washington, D.C., United States [doi>10.1145/57167.57204]
• LC91: Michael Cohen , Lester F. Ludwig, Multidimensional audio window management, International Journal of Man-Machine Studies, v.34 n.3, p.319-336, March 1991 [doi>10.1016/0020-7373(91)90023-Z]
• LPC90: Lester F. Ludwig , Michael Cohen , Natalio Pincever, Extending the Notion of a Window System to Audio, Computer, v.23 n.8, p.66-72, August 1990 [doi>10.1109/2.56873]
• ME91: Elizabeth Mynatt and Keith Edwards. New metaphors for nonvisual interfaces. In Extraordinary Human-Computer Interaction, 1991. Draft chapter accepted for upcoming book.
• ME92: Elizabeth Mynatt and W. Keith Edwards. The Mercator Environment: A Nonvisual Interface to X Windows and Unix Workstations. GVU Tech Report GIT-GVU-92-05. February 1992.
• Pet91: Chris D. Peterson. Editres-a graphical resource editor for x toolkit applications. In Conference Proceedings, Fifth Annual X Technical Conference, Boston, Massachusetts, January, 1991.
• Sch87: Robert W. Scheifler. X window system protocol specification, version I I. Massachusetts Institute of Technology, Cambridge, Massachusetts, and Digital Equipment Corporation, Maynard, Massachusetts, 1987.
• Van89: G.C. Vanderheiclen. Nonvisual alternative display techniques for output from graphicsbased computers. Journal of Visual Impairment and Blindness, 1989.
• WWF88: Elizabeth M. Wenzel , Frederic L. Wightman , Scott H. Foster, Development of a three-dimensional auditory display system, ACM SIGCHI Bulletin, v.20 n.2, p.52-57, Oct. 1988 [doi>10.1145/54386.54405]
• Yor89: Bryant W. York, editor. Final Report of the Boston University Workshop on Computers and Persons with Disabilities, 1989.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques

          Nouns: X-Window
  D.3 PROGRAMMING LANGUAGES
      D.3.2 Language Classifications

          Nouns: C++

  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Audio input/output

General Terms:
Design, Human Factors