Handle Flags: efficient and flexible selections for inking applications

ABSTRACT

There are a number of challenges associated with content selection in pen-based interfaces. Supplementary buttons to enter a selection mode may not be available, and selections may require a careful and error prone lasso stroke. In this paper we describe the design and evaluation of Handle Flags, a new localized technique used to select and perform commands on ink strokes in pen-operated interfaces. When the user positions the pen near an ink stroke, Handle Flags are displayed for the potential selections that the ink stroke could belong to (such as proximal strokes comprising a word or drawing). Tapping the handle allows the user to access the corresponding selection, without requiring a complex lasso stroke. Our studies show that Handle Flags offer significant benefits in comparison to traditional techniques, and are a promising technique for pen-based applications.

REFERENCES

• 1: Johnny Accot , Shumin Zhai, Beyond Fitts' law: models for trajectory-based HCI tasks, Proceedings of the SIGCHI conference on Human factors in computing systems, p.295-302, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258760]
• 2: Johnny Accot , Shumin Zhai, More than dotting the i's --- foundations for crossing-based interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503390]
• 3: Alvarado, C. (2004). Sketch Recognition User Interfaces: Guidelines for Design and Development. AAAI Fall Symposium on Intelligent Pen-based Interfaces. 8--14.
• 4: Buxton, W., Fiume, E., Hill, R., Lee, A. and Woo, C. (1983). Continuous Hand-Gesture Driven Input. Graphic Interface. 191--195.
• 5: Deming, K. and Lank, E. (2004). Managing Ambiguous Intention in Mode Inferencing. Proc of the AAAI Fall Symposium Series. 49--54.
• 6: George W. Fitzmaurice , Ravin Balakrishnan , Gordon Kurtenbach , Bill Buxton, An exploration into supporting artwork orientation in the user interface, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.167-174, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303033]
• 7: George Fitzmaurice , Azam Khan , Robert Pieké , Bill Buxton , Gordon Kurtenbach, Tracking menus, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.71-79, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964704]
• 8: Tovi Grossman , Ken Hinckley , Patrick Baudisch , Maneesh Agrawala , Ravin Balakrishnan, Hover widgets: using the tracking state to extend the capabilities of pen-operated devices, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124898]
• 9: Guiard, Y. (1987). Asymmetric division of labor in human skilled bimanual action: The kinematic chain as a model. Journal of Motor Behavior. 19(4): 486--517.
• 10: Ken Hinckley , Patrick Baudisch , Gonzalo Ramos , Francois Guimbretiere, Design and analysis of delimiters for selection-action pen gesture phrases in scriboli, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055035]
• 11: Ken Hinckley , Francois Guimbretiere , Maneesh Agrawala , Georg Apitz , Nicholas Chen, Phrasing techniques for multi-stroke selection gestures, Proceedings of Graphics Interface 2006, June 07-09, 2006, Quebec, Canada
• 12: Ken Hinckley , Francois Guimbretiere , Patrick Baudisch , Raman Sarin , Maneesh Agrawala , Ed Cutrell, The springboard: multiple modes in one spring-loaded control, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124801]
• 13: Gordon Kurtenbach , William Buxton, Issues in combining marking and direct manipulation techniques, Proceedings of the 4th annual ACM symposium on User interface software and technology, p.137-144, November 11-13, 1991, Hilton Head, South Carolina, United States [doi>10.1145/120782.120797]
• 14: Lank, E. and Saund, E. (2005). Sloppy Selection: Providing an Accurate Interpretation of Imprecise Stylus Selection Gestures. Computers and Graphics. 29(4): 490--500.
• 15: Yang Li , Ken Hinckley , Zhiwei Guan , James A. Landay, Experimental analysis of mode switching techniques in pen-based user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055036]
• 16: Jennifer Mankoff , Scott E. Hudson , Gregory D. Abowd, Interaction techniques for ambiguity resolution in recognition-based interfaces, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.11-20, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354407]
• 17: Sachi Mizobuchi , Michiaki Yasumura, Tapping vs. circling selections on pen-based devices: evidence for different performance-shaping factors, Proceedings of the SIGCHI conference on Human factors in computing systems, p.607-614, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985769]
• 18: Thomas P. Moran , Patrick Chiu , William van Melle, Pen-based interaction techniques for organizing material on an electronic whiteboard, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.45-54, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263508]
• 19: Elizabeth D. Mynatt , Takeo Igarashi , W. Keith Edwards , Anthony LaMarca, Flatland: new dimensions in office whiteboards, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.346-353, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303108]
• 20: Norman, D. A. (1981). Categorization of Action Slips. Psychology Review. 88(1): 1--15.
• 21: Réjean Plamondon , Sargur N. Srihari, On-Line and Off-Line Handwriting Recognition: A Comprehensive Survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.1, p.63-84, January 2000 [doi>10.1109/34.824821]
• 22: Gonzalo Ramos , George Robertson , Mary Czerwinski , Desney Tan , Patrick Baudisch , Ken Hinckley , Maneesh Agrawala, Tumble! Splat! helping users access and manipulate occluded content in 2D drawings, Proceedings of the working conference on Advanced visual interfaces, May 23-26, 2006, Venezia, Italy [doi>10.1145/1133265.1133351]
• 23: Xiangshi Ren , Shinju Moriya, Improving selection performance on pen-based systems: a study of pen-based interaction for selection tasks, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.3, p.384-416, Sept. 2000 [doi>10.1145/355324.355328]
• 24: Eric Saund , David Fleet , Daniel Larner , James Mahoney, Perceptually-supported image editing of text and graphics, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.183-192, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964717]
• 25: Eric Saund , Edward Lank, Stylus input and editing without prior selection of mode, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.213-216, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964720]
• 26: Saund, E., Mahoney, J., Fleet, D., Larner, D. and Lank, E. (2002). Perceptual Organization as a Foundation for Intelligent Sketch Editing. AAAI Symposium on Sketch Understanding. 118--125.
• 27: Eric Saund , Thomas P. Moran, A perceptually-supported sketch editor, Proceedings of the 7th annual ACM symposium on User interface software and technology, p.175-184, November 02-04, 1994, Marina del Rey, California, United States [doi>10.1145/192426.192494]
• 28: Abigail J. Sellen , Gordon P. Kurtenbach , William A. S. Buxton, The prevention of mode errors through sensory feedback, Human-Computer Interaction, v.7 n.2, p.141-164, June 1992 [doi>10.1207/s15327051hci0702_1]
• 29: Robert Zeleznik , Timothy Miller, Fluid inking: augmenting the medium of free-form inking with gestures, Proceedings of Graphics Interface 2006, June 07-09, 2006, Quebec, Canada

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Keywords:
Handle Flag, ink, lasso, pen input, selection