Dynamically adapting GUIs to diverse input devices

ABSTRACT

Many of today's desktop applications are designed for use with a pointing device and keyboard. Someone with a disability, or in a unique environment, may not be able to use one or both of these devices. We have developed an approach for automatically modifying desktop applications to accommodate a variety of input alternatives as well as a demonstration implementation, the Input Adapter Tool (IAT). Our work is differentiated from past work by our focus on input adaptation (such as adapting a paint program to work without a pointing device) rather than output adaptation (such as adapting web pages to work on a cellphone). We present an analysis showing how different common interactive elements and navigation techniques can be adapted to specific input modalities. We also describe IAT, which supports a subset of these adaptations, and illustrate how it adapts different inputs to two applications, a paint program and a form entry program.

REFERENCES

• 1: Apple Computer Inc., MacIntosh Human Interface Guidelines, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1992
• 2: Lior Berry , Lyn Bartram , Kellogg S. Booth, Role-based control of shared application views, Proceedings of the 18th annual ACM symposium on User interface software and technology, October 23-26, 2005, Seattle, WA, USA [doi>10.1145/1095034.1095039]
• 3: William Buxton, A three-state model of graphical input, Proceedings of the IFIP TC13 Third Interational Conference on Human-Computer Interaction, p.449-456, August 27-31, 1990
• 4: Orkut Buyukkokten , Hector Garcia-Molina , Andreas Paepcke , Terry Winograd, Power browser: efficient Web browsing for PDAs, Proceedings of the SIGCHI conference on Human factors in computing systems, p.430-437, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332470]
• 5: Stuart K. Card , Jock D. Mackinlay , George G. Robertson, A morphological analysis of the design space of input devices, ACM Transactions on Information Systems (TOIS), v.9 n.2, p.99-122, April 1991 [doi>10.1145/123078.128726]
• 6: Bay-Wei Chang , Jock D. Mackinlay , Polle T. Zellweger , Takeo Igarashi, A negotiation architecture for fluid documents, Proceedings of the 11th annual ACM symposium on User interface software and technology, p.123-132, November 01-04, 1998, San Francisco, California, United States [doi>10.1145/288392.288585]
• 7: Liwei Dai , Rich Goldman , Andrew Sears , Jeremy Lozier, Speech-based cursor control: a study of grid-based solutions, Proceedings of the 6th international ACM SIGACCESS conference on Computers and accessibility, October 18-20, 2004, Atlanta, GA, USA
• 8: Dan R. Olsen, Jr. , Scott E. Hudson , Thom Verratti , Jeremy M. Heiner , Matt Phelps, Implementing interface attachments based on surface representations, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.191-198, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303038]
• 9: W. Keith Edwards , Elizabeth D. Mynatt, An architecture for transforming graphical interfaces, Proceedings of the 7th annual ACM symposium on User interface software and technology, p.39-47, November 02-04, 1994, Marina del Rey, California, United States [doi>10.1145/192426.192443]
• 10: P. Fitts. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47(6), pages 381--391, 1954.
• 11: James D. Foley , Victor L. Wallace , Peggy Chan, The human factors of computer graphics interaction techniques, IEEE Computer Graphics and Applications, v.4 n.11, p.13-48, Nov. 1984
• 12: Susanne Jul , George W. Furnas, From brains to branch points: cognitive constraints in navigational design, 2004
• 13: Hesham M. Kamel , James A. Landay, Sketching images eyes-free: a grid-based dynamic drawing tool for the blind, Proceedings of the fifth international ACM conference on Assistive technologies, July 08-10, 2002, Edinburgh, Scotland [doi>10.1145/638249.638258]
• 14: Donald E. Knuth, The art of computer programming, volume 1 (3rd ed.): fundamental algorithms, Addison Wesley Longman Publishing Co., Inc., Redwood City, CA, 1997
• 15: Jennifer Mankoff , Anind Dey , Udit Batra , Melody Moore, Web accessibility for low bandwidth input, Proceedings of the fifth international ACM conference on Assistive technologies, July 08-10, 2002, Edinburgh, Scotland [doi>10.1145/638249.638255]
• 16: Yoshiyuki Mihara , Etsuya Shibayama , Shin Takahashi, The migratory cursor: accurate speech-based cursor movement by moving multiple ghost cursors using non-verbal vocalizations, Proceedings of the 7th international ACM SIGACCESS conference on Computers and accessibility, October 09-12, 2005, Baltimore, MD, USA [doi>10.1145/1090785.1090801]
• 17: Melody Moore , Philip Kennedy , Elizabeth Mynatt , Jennifer Mankoff, Nudge and shove: frequency thresholding for navigation in direct brain-computer interfaces, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634280]
• 18: Brad A. Myers, Using handhelds and PCs together, Communications of the ACM, v.44 n.11, p.34-41, Nov. 2001 [doi>10.1145/384150.384159]
• 19: A. Newell and P. Gregor. Human computer interaction for people with disabilities. Handbook of Human-Computer Interaction, pages 813--824. Elsevier Science Publishers, 1988.
• 20: Jeffrey Nichols , Brad A. Myers , Michael Higgins , Joseph Hughes , Thomas K. Harris , Roni Rosenfeld , Mathilde Pignol, Generating remote control interfaces for complex appliances, Proceedings of the 15th annual ACM symposium on User interface software and technology, October 27-30, 2002, Paris, France [doi>10.1145/571985.572008]
• 21: Dan R. Olsen, Jr. , Travis Nielsen, Laser pointer interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, p.17-22, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365030]
• 22: Dan R. Olsen, Jr. , Sean Jefferies , Travis Nielsen , William Moyes , Paul Fredrickson, Cross-modal interaction using XWeb, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.191-200, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354764]
• 23: Neil G. Scott , Isabelle Gingras, The total access system, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634078]
• 24: Jingtao Wang , Jennifer Mankoff, Theoretical and architectural support for input device adaptation, Proceedings of the 2003 conference on Universal usability, November 10-11, 2003, Vancouver, British Columbia, Canada

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.4 COMPUTERS AND SOCIETY
      K.4.2 Social Issues
          Subjects: Assistive technologies for persons with disabilities

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

General Terms:
Design, Human Factors

Keywords:
accessibility, interaction techniques, toolkits