GestureBar: improving the approachability of gesture-based interfaces

ABSTRACT

GestureBar is a novel, approachable UI for learning gestural interactions that enables a walk-up-and-use experience which is in the same class as standard menu and toolbar interfaces. GestureBar leverages the familiar, clean look of a common toolbar, but in place of executing commands, richly discloses how to execute commands with gestures, through animated images, detail tips and an out-of-document practice area. GestureBar's simple design is also general enough for use with any recognition technique and for integration with standard, non-gestural UI components. We evaluate GestureBar in a formal experiment showing that users can perform complex, ecologically valid tasks in a purely gestural system without training, introduction, or prior gesture experience when using GestureBar, discovering and learning a high percentage of the gestures needed to perform the tasks optimally, and significantly outperforming a state of the art crib sheet. The relative contribution of the major design elements of GestureBar is also explored. A second experiment shows that GestureBar is preferred to a basic crib sheet and two enhanced crib sheet variations.

REFERENCES

• 1: Alvarado, C. Sketch Recognition User Interfaces: Guidelines for Design and Development. In Proc. of AAAI Fall Symposium on Intelligent Pen-based Interfaces, (2004).
• 2: Christine Alvarado , Randall Davis, SketchREAD: a multi-domain sketch recognition engine, Proceedings of the 17th annual ACM symposium on User interface software and technology, October 24-27, 2004, Santa Fe, NM, USA [doi>10.1145/1029632.1029637]
• 3: Olivier Bau , Wendy E. Mackay, OctoPocus: a dynamic guide for learning gesture-based command sets, Proceedings of the 21st annual ACM symposium on User interface software and technology, October 19-22, 2008, Monterey, CA, USA [doi>10.1145/1449715.1449724]
• 4: Buxton, W. Chunking and phrasing and the design of human-computer dialogues. IFIP World Computer Congress'86, 475--480.
• 5: Buxton, W., Fiume, E., Hill, R., Lee, A., Woo, C. Continuous Hand-Gesture Driven Input. Graphics Interface '83, 191--195.
• 6: Forsberg, A, Holden, L., Miller, T., and Zeleznik, R. The Music Notepad, Brown University (2005).
• 7: Tovi Grossman , Pierre Dragicevic , Ravin Balakrishnan, Strategies for accelerating on-line learning of hotkeys, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240865]
• 8: Tovi Grossman , Ken Hinckley , Patrick Baudisch , Maneesh Agrawala , Ravin Balakrishnan, Hover widgets: using the tracking state to extend the capabilities of pen-operated devices, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124898]
• 9: Ken Hinckley , Patrick Baudisch , Gonzalo Ramos , Francois Guimbretiere, Design and analysis of delimiters for selection-action pen gesture phrases in scriboli, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055035]
• 10: Ken Hinckley , Shengdong Zhao , Raman Sarin , Patrick Baudisch , Edward Cutrell , Michael Shilman , Desney Tan, InkSeine: In Situ search for active note taking, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240666]
• 11: Holm, S. A Simple Sequentially Rejective Multiple Test Procedure. Scandinavian Journal of Statistics, 6 (1979), 60--65.
• 12: Jason I. Hong , James A. Landay, SATIN: a toolkit for informal ink-based applications, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.63-72, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354412]
• 13: Hse, H. and Newton, A. Recognition and beautification of multi-stroke symbols in digital ink. Computers&Graphics, 29, 4 (August 2005), 533--546.
• 14: Takeo Igarashi , Satoshi Matsuoka , Sachiko Kawachiya , Hidehiko Tanaka, Interactive beautification: a technique for rapid geometric design, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.105-114, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263525]
• 15: Kurtenbach, G., Moran, T. P. and Buxton, W. Contextual Animation of Gestural Commands. Graphics Interface '94, 83--90.
• 16: A. Chris Long, Jr. , James A. Landay , Lawrence A. Rowe , Joseph Michiels, Visual similarity of pen gestures, Proceedings of the SIGCHI conference on Human factors in computing systems, p.360-367, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332458]
• 17: Jennifer Mankoff , Scott E. Hudson , Gregory D. Abowd, Providing integrated toolkit-level support for ambiguity in recognition-based interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, p.368-375, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332459]
• 18: Microsoft, Inc. Office 2007. 2006.
• 19: Microsoft, Inc. Windows Vista. 2006.
• 20: Mouse Gestures Add-on for the Mozilla Firefox Web Browser. http://www.mousegestures.org/.
• 21: Palm, Inc. Graffiti character recognizer.
• 22: Peter G. Polson , Clayton H. Lewis, Theory-based design for easily learned interfaces, Human-Computer Interaction, v.5 n.2, p.191-220, June 1990 [doi>10.1207/s15327051hci0502&3_3]
• 23: Dean Rubine, Specifying gestures by example, Proceedings of the 18th annual conference on Computer graphics and interactive techniques, p.329-337, July 1991
• 24: Mark A. Tapia , Gordon Kurtenbach, Some design refinements and principles on the appearance and behavior of marking menus, Proceedings of the 8th annual ACM symposium on User interface and software technology, p.189-195, November 15-17, 1995, Pittsburgh, Pennsylvania, United States [doi>10.1145/215585.215973]
• 25: Jacob O. Wobbrock , Andrew D. Wilson , Yang Li, Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes, Proceedings of the 20th annual ACM symposium on User interface software and technology, October 07-10, 2007, Newport, Rhode Island, USA [doi>10.1145/1294211.1294238]
• 26: Robert C. Zeleznik , Andrew Bragdon , Chu-Chi Liu , Andrew Forsberg, Lineogrammer: creating diagrams by drawing, Proceedings of the 21st annual ACM symposium on User interface software and technology, October 19-22, 2008, Monterey, CA, USA [doi>10.1145/1449715.1449741]
• 27: Robert Zeleznik , Timothy Miller, Fluid inking: augmenting the medium of free-form inking with gestures, Proceedings of Graphics Interface 2006, June 07-09, 2006, Quebec, Canada
• 28: Shengdong Zhao , Maneesh Agrawala , Ken Hinckley, Zone and polygon menus: using relative position to increase the breadth of multi-stroke marking menus, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124933]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

General Terms:
Human Factors

Keywords:
approachability, disclosure, gestures, learning, pen