Support for input adaptability in the ICON toolkit

ABSTRACT

In this paper, we introduce input adaptability as the ability of an application to exploit alternative sets of input devices effectively and offer users a way of adapting input interaction to suit their needs. We explain why input adaptability must be seriously considered today and show how it is poorly supported by current systems, applications and tools. We then describe ICon (Input Configurator), an input toolkit that allows interactive applications to achieve a high level of input adaptability. We present the software architecture behind ICon then the toolkit itself, and give several examples of non-standard interaction techniques that are easy to build and modify using ICon's graphical editor while being hard or impossible to support using regular GUI toolkits.

REFERENCES

• 1: Adobe Creative Team, Adobe Photoshop 7.0: Classroom in a Book, Adobe Press; ISBN: 0-321-11562-7
• 2: Avid Inc. Channel Developer's Kit, Softimage Inc., 2000
• 3: Ravin Balakrishnan , George Fitzmaurice , Gordon Kurtenbach , William Buxton, Digital tape drawing, Proceedings of the 12th annual ACM symposium on User interface software and technology, p.161-169, November 07-10, 1999, Asheville, North Carolina, United States [doi>10.1145/320719.322598]
• 4: Rémi Bastide , David Navarre , Philippe Palanque, A model-based tool for interactive prototyping of highly interactive applications, CHI '02 extended abstracts on Human factors in computing systems, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/506443.506457]
• 5: Thomas Baudel, A mark-based interaction paradigm for free-hand drawing, Proceedings of the 7th annual ACM symposium on User interface software and technology, p.185-192, November 02-04, 1994, Marina del Rey, California, United States [doi>10.1145/192426.192496]
• 6: Benjamin B. Bederson , Jon Meyer , Lance Good, Jazz: an extensible zoomable user interface graphics toolkit in Java, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.171-180, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354754]
• 7: Berry, G. The Esterel v5 language primer. Tech report, april 1999. http://www-sop.inria.fr/meije/esterel/doc/main-papers.html.
• 8: Eric A. Bier , Steven Freeman, MMM: a user interface architecture for shared editors on a single screen, Proceedings of the 4th annual ACM symposium on User interface software and technology, p.79-86, November 11-13, 1991, Hilton Head, South Carolina, United States [doi>10.1145/120782.120791]
• 9: Richard A. Bolt, “Put-that-there”: Voice and gesture at the graphics interface, Proceedings of the 7th annual conference on Computer graphics and interactive techniques, p.262-270, July 14-18, 1980, Seattle, Washington, United States
• 10: Alan Hamilton Borning, Thinglab--a constraint-oriented simulation laboratory., 1979
• 11: Buxton, W. There's More to Interaction than Meets the Eye: Some Issues in Manual Input. In Norman, D. A. and Draper, S. W. (Eds.), (1986), User Centered System Design: New Perspectives on Human-Computer Interaction. Lawrence Erlbaum Associates, Hillsdale, New Jersey, 319--337.
• 12: Eamon Doherty , Gilbert Cockton , Chris Bloor , Dennis Benigno, Improving the performance of the cyberlink mental interface with “yes / no program”, Proceedings of the SIGCHI conference on Human factors in computing systems, p.69-76, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365038]
• 13: Eric A. Bier , Maureen C. Stone , Ken Fishkin , William Buxton , Thomas Baudel, A taxonomy of see-through tools, Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, p.358-364, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/191666.191786]
• 14: Robert Englander, Developing Java beans, O'Reilly & Associates, Inc., Sebastopol, CA, 1997
• 15: Esteban, O., Chatty, S., Palanque, P. Whizz'ed : a visual environment for building highly interactive software. In Proc. of IFIP INTERACT'95 : Human-Computer Interaction, pages 121--126, 1995.
• 16: Paula Ferguson, The X11 input extension, The X Resource
• 17: James D. Foley , Andries van Dam , Steven K. Feiner , John F. Hughes, Computer graphics: principles and practice (2nd ed.), Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1990
• 18: Saul Greenberg , Chester Fitchett, Phidgets: easy development of physical interfaces through physical widgets, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502388]
• 19: Halbwachs, N., Caspi, P., Raymond, P. and Pilaud, D. The synchronous data-flow programming language LUSTRE. In Proc. of the IEEE, volume 79, September 1991.
• 20: Honeywell, S. Quake III Arena: Prima's Official Strategy Guide. Prima Publishing, 1999.
• 21: Hourcade, J.P. and Bederson, B.B. Architecture and Implementation of a Java Package for Multiple Input Devices (MID), Human-Computer Interaction Laboratory, University of Maryland, College Park, MD 20742, USA, 1999.
• 22: Jason I. Hong , James A. Landay, SATIN: a toolkit for informal ink-based applications, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.63-72, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354412]
• 23: Dan Ingalls , Scott Wallace , Yu-Ying Chow , Frank Ludolph , Ken Doyle, Fabrik: a visual programming environment, Conference proceedings on Object-oriented programming systems, languages and applications, p.176-190, September 25-30, 1988, San Diego, California, United States
• 24: Robert J. K. Jacob , Leonidas Deligiannidis , Stephen Morrison, A software model and specification language for non-WIMP user interfaces, ACM Transactions on Computer-Human Interaction (TOCHI), v.6 n.1, p.1-46, March 1999 [doi>10.1145/310641.310642]
• 25: Brian Knep , Craig Hayes , Rick Sayre , Tom Williams, Dinosaur input device, Proceedings of the SIGCHI conference on Human factors in computing systems, p.304-309, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223943]
• 26: LCS/Telegraphics. The Wintab Developers' Kit, http://www.pointing.com/WINTAB.HTM, 1999.
• 27: Mackay, W. E. Which Interaction Technique Works When? Floating Palettes, Marking Menus and Toolglasses Support Different Task Strategies. in Proc. International Conference on Advanced Visual Interfaces (AVI 2002) , ACM Press, pages 203--208, April, 2002.
• 28: Jennifer Mankoff , Scott E. Hudson , Gregory D. Abowd, Providing integrated toolkit-level support for ambiguity in recognition-based interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, p.368-375, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332459]
• 29: McLoone, H., Hinckley, K. & Cutrell, E. Bimanual Interaction on the Microsoft Office Keyboard. In Rauterberg,M., Menozzi, M, & Wesson, J. (Eds.), Human-Computer Interaction INTERACT '03, Züürich, September 2003, pp. 49--56.
• 30: Microsoft Corporation, DirectX 9 Graphics Programmers Guide, Microsoft Press International; ISBN: 0735616531, March 2001.
• 31: Microsoft Corporation, Microsoft Active Accessibility SDK 1.2, Microsoft Corporation 1999.
• 32: Melody Moore , Philip Kennedy , Elizabeth Mynatt , Jennifer Mankoff, Nudge and shove: frequency thresholding for navigation in direct brain-computer interfaces, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634280]
• 33: Brad A. Myers, A new model for handling input, ACM Transactions on Information Systems (TOIS), v.8 n.3, p.289-320, July 1990 [doi>10.1145/98188.98204]
• 34: Kurt Partridge , Saurav Chatterjee , Vibha Sazawal , Gaetano Borriello , Roy Want, TiltType: accelerometer-supported text entry for very small devices, Proceedings of the 15th annual ACM symposium on User interface software and technology, October 27-30, 2002, Paris, France [doi>10.1145/571985.572013]
• 35: Ken Perlin, Quikwriting: continuous stylus-based text entry, Proceedings of the 11th annual ACM symposium on User interface software and technology, p.215-216, November 01-04, 1998, San Francisco, California, United States [doi>10.1145/288392.288613]
• 36: Jenny Preece , Yvonne Rogers , Helen Sharp , David Benyon, Human-Computer Interaction, Addison-Wesley Longman Ltd., Essex, UK, 1994
• 37: Jun Rekimoto, Tilting operations for small screen interfaces, Proceedings of the 9th annual ACM symposium on User interface software and technology, p.167-168, November 06-08, 1996, Seattle, Washington, United States [doi>10.1145/237091.237115]
• 38: Daniel Salber , Anind K. Dey , Gregory D. Abowd, The context toolkit: aiding the development of context-enabled applications, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.434-441, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303126]
• 39: Sense8 Corp. The World Toolkit Manual, Sense8, 1999.
• 40: Sun Microsystems Inc. Java Speech API Programmer's Guide, http://java.sun.com/ 1998.
• 41: Tyson R. Henry , Scott E. Hudson , Gary L. Newell, Integrating gesture and snapping into a user interface toolkit, Proceedings of the 3rd annual ACM SIGGRAPH symposium on User interface software and technology, p.112-122, October 03-05, 1990, Snowbird, Utah, United States [doi>10.1145/97924.97938]
• 42: Virtools dev. Virtools SA, 2001. http://www.virtools.com/.
• 43: Westergaard M. Supporting Multiple Pointing Devices in Microsoft Windows. Microsoft Summer Workshop for Faculty and PhDs. Cambridge, England, September 2002.
• 44: Dragicevic, P. and Fekete, J.-D., Input Device Selection and Interaction Configuration with ICon, Proc. of IHM-HCI 2001,Blandford, A.; Vanderdonckt, J.; Gray, P., (Eds.): People and Computers XV - Interaction without Frontiers, Lille, France, Springer Verlag, pp.543--448.
• 45: Thevenin, D., Coutaz, J., Plasticity of User Interfaces: Framework and Research Agenda. In Proc. of Interact'99, September 3, 1999. Chapman & Hall, London, pp. 110--117

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: User interface management systems (UIMS)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen); Prototyping

General Terms:
Design

Keywords:
adaptability, input devices, interaction techniques, toolkits, visual programming