OctoPocus: a dynamic guide for learning gesture-based command sets

ABSTRACT

We describe OctoPocus, an example of a dynamic guide that combines on-screen feedforward and feedback to help users learn, execute and remember gesture sets. OctoPocus can be applied to a wide range of single-stroke gestures and recognition algorithms and helps users progress smoothly from novice to expert performance. We provide an analysis of the design space and describe the results of two experi-ments that show that OctoPocus is significantly faster and improves learning of arbitrary gestures, compared to con-ventional Help menus. It can also be adapted to a mark-based gesture set, significantly improving input time compared to a two-level, four-item Hierarchical Marking menu.

REFERENCES

• 1: Peter Agar , Kevin Novins, Polygon recognition in sketch-based interfaces with immediate and continuous feedback, Proceedings of the 1st international conference on Computer graphics and interactive techniques in Australasia and South East Asia, February 11-14, 2003, Melbourne, Australia [doi>10.1145/604471.604500]
• 2: James Arvo , Kevin Novins, Fluid sketches: continuous recognition and morphing of simple hand-drawn shapes, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.73-80, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354413]
• 3: James Arvo , Kevin Novins, Fluid sketching of directed graphs, Proceedings of the 7th Australasian User interface conference, p.81-86, January 16-19, 2006, Hobart, Australia
• 4: Daniel Avrahami , Scott E. Hudson , Thomas P. Moran , Brian D. Williams, Guided gesture support in the paper PDA, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502385]
• 5: Blinkenstorfer, C. (1995) Graffiti. Pen Computing, p. 30--31.
• 6: J. Callahan , D. Hopkins , M. Weiser , B. Shneiderman, An empirical comparison of pie vs. linear menus, Proceedings of the SIGCHI conference on Human factors in computing systems, p.95-100, May 15-19, 1988, Washington, D.C., United States [doi>10.1145/57167.57182]
• 7: Xiang Cao , Shumin Zhai, Modeling human performance of pen stroke gestures, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240850]
• 8: Stéphane Chatty , Patrick Lecoanet, Pen computing for air traffic control, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.87-94, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238436]
• 9: Jacob Eisenstein , Wendy E. Mackay, Interacting with communication appliances: an evaluation of two computer vision-based selection techniques, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124938]
• 10: David Goldberg , Cate Richardson, Touch-typing with a stylus, Proceedings of the INTERACT '93 and CHI '93 conference on Human factors in computing systems, p.80-87, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169093]
• 11: Tovi Grossman , Ken Hinckley , Patrick Baudisch , Maneesh Agrawala , Ravin Balakrishnan, Hover widgets: using the tracking state to extend the capabilities of pen-operated devices, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124898]
• 12: François Guimbretiére , Terry Winograd, FlowMenu: combining command, text, and data entry, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.213-216, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354778]
• 13: Jason I. Hong , James A. Landay, SATIN: a toolkit for informal ink-based applications, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.63-72, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354412]
• 14: Heloise Hse , Michael Shilman , A. Richard Newton, Robust sketched symbol fragmentation using templates, Proceedings of the 9th international conference on Intelligent user interfaces, January 13-16, 2004, Funchal, Madeira, Portugal [doi>10.1145/964442.964472]
• 15: Giancarlo Iannizzotto , Lorenzo Vita, A Multiscale Turning Angle Representation of Object Shapes for Image Retrieval, Proceedings of the Third International Conference on Visual Information and Information Systems, p.609-616, June 02-04, 1999
• 16: Takeo Igarashi , Satoshi Matsuoka , Sachiko Kawachiya , Hidehiko Tanaka, Interactive beautification: a technique for rapid geometric design, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.105-114, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263525]
• 17: Takeo Igarashi , Sachiko Kawachiya , Hidehiko Tanaka , Satoshi Matsuoka, Pegasus: a drawing system for rapid geometric design, CHI 98 conference summary on Human factors in computing systems, p.24-25, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/286498.286511]
• 18: Gordon Kurtenbach , William Buxton, The limits of expert performance using hierarchic marking menus, Proceedings of the INTERACT '93 and CHI '93 conference on Human factors in computing systems, p.482-487, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169426]
• 19: Gordon Paul Kurtenbach, The design and evaluation of marking menus, University of Toronto, Toronto, Ont., Canada, 1993
• 20: Kurtenbach, G. & Moran, T. (1994) Contextual Animation of Gestural Commands. In Proc. Eurographics Computer Graphics Forum. Vol. 13(5), 305--314.
• 21: Junfeng Li , Xiwen Zhang , Xiang Ao , Guozhong Dai, Sketch recognition with continuous feedback based on incremental intention extraction, Proceedings of the 10th international conference on Intelligent user interfaces, January 10-13, 2005, San Diego, California, USA [doi>10.1145/1040830.1040866]
• 22: Chunyuan Liao , François Guimbretière , Ken Hinckley, PapierCraft: a command system for interactive paper, Proceedings of the 18th annual ACM symposium on User interface software and technology, October 23-26, 2005, Seattle, WA, USA [doi>10.1145/1095034.1095074]
• 23: Allan Christian Long, Jr. , James A. Landay , Lawrence A. Rowe, Implications for a gesture design tool, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.40-47, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.302985]
• 24: A. Chris Long, Jr. , James A. Landay , Lawrence A. Rowe , Joseph Michiels, Visual similarity of pen gestures, Proceedings of the SIGCHI conference on Human factors in computing systems, p.360-367, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332458]
• 25: Wendy E. Mackay , Anne-Laure Fayard , Laurent Frobert , Lionel Médini, Reinventing the familiar: exploring an augmented reality design space for air traffic control, Proceedings of the SIGCHI conference on Human factors in computing systems, p.558-565, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274719]
• 26: Jennifer Mankoff , Scott E. Hudson , Gregory D. Abowd, Interaction techniques for ambiguity resolution in recognition-based interfaces, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.11-20, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354407]
• 27: Michael Moyle , Andy Cockburn, The design and evaluation of a flick gesture for 'back' and 'forward' in web browsers, Proceedings of the Fourth Australasian user interface conference on User interfaces 2003, p.39-46, February 01, 2003, Adelaide, Australia
• 28: Ken Perlin, Quikwriting: continuous stylus-based text entry, Proceedings of the 11th annual ACM symposium on User interface software and technology, p.215-216, November 01-04, 1998, San Francisco, California, United States [doi>10.1145/288392.288613]
• 29: Dean Rubine, Specifying gestures by example, Proceedings of the 18th annual conference on Computer graphics and interactive techniques, p.329-337, July 1991
• 30: Dean Rubine, Combining gestures and direct manipulation, Proceedings of the SIGCHI conference on Human factors in computing systems, p.659-660, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.143072]
• 31: Jacob O. Wobbrock , Andrew D. Wilson , Yang Li, Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes, Proceedings of the 20th annual ACM symposium on User interface software and technology, October 07-10, 2007, Newport, Rhode Island, USA [doi>10.1145/1294211.1294238]
• 32: Shengdong Zhao , Ravin Balakrishnan, Simple vs. compound mark hierarchical marking menus, Proceedings of the 17th annual ACM symposium on User interface software and technology, October 24-27, 2004, Santa Fe, NM, USA [doi>10.1145/1029632.1029639]
• 33: Shengdong Zhao , Maneesh Agrawala , Ken Hinckley, Zone and polygon menus: using relative position to increase the breadth of multi-stroke marking menus, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124933]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: User interfaces

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

General Terms:
Design, Experimentation, Human Factors

Keywords:
dynamic guides, feedback, feedforward, gesture recognition, mouse input, octopocus, pen input