Shift: a technique for operating pen-based interfaces using touch

ABSTRACT

Retrieving the stylus of a pen-based device takes time and requires a second hand. Especially for short intermittent interactions many users therefore choose to use their bare fingers. Although convenient, this increases targeting times and error rates. We argue that the main reasons are the occlusion of the target by the user's finger and ambiguity about which part of the finger defines the selection point. We propose a pointing technique we call Shift that is designed to address these issues. When the user touches the screen, Shift creates a callout showing a copy of the occluded screen area and places it in a non-occluded location. The callout also shows a pointer representing the selection point of the finger. Using this visual feedback, users guide the pointer into the target by moving their finger on the screen surface and commit the target acquisition by lifting the finger. Unlike existing techniques, Shift is only invoked when necessary--over large targets no callout is created and users enjoy the full performance of an unaltered touch screen. We report the results of a user study showing that with Shift participants can select small targets with much lower error rates than an unaided touch screen and that Shift is faster than Offset Cursor for larger targets.

REFERENCES

• 1: Pär-Anders Albinsson , Shumin Zhai, High precision touch screen interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642631]
• 2: Ravin Balakrishnan , I. Scott MacKenzie, Performance differences in the fingers, wrist, and forearm in computer input control, Proceedings of the SIGCHI conference on Human factors in computing systems, p.303-310, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258764]
• 3: Baudisch, P., Cutrell, E., Robbins, D., Czerwinski, M., Tandler, P. Bederson, B., Zierlinger, A. (2003). Drag-and-Pop and Drag-and-Pick: Techniques for Accessing Remote Screen Content on Touch and Pen--operated Systems. In Proc. of Interact'03, 57--64.
• 4: Patrick Baudisch , Desney Tan , Maxime Collomb , Dan Robbins , Ken Hinckley , Maneesh Agrawala , Shengdong Zhao , Gonzalo Ramos, Phosphor: explaining transitions in the user interface using afterglow effects, Proceedings of the 19th annual ACM symposium on User interface software and technology, October 15-18, 2006, Montreux, Switzerland [doi>10.1145/1166253.1166280]
• 5: Hrvoje Benko , Andrew D. Wilson , Patrick Baudisch, Precise selection techniques for multi-touch screens, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124963]
• 6: Renaud Blanch , Yves Guiard , Michel Beaudouin-Lafon, Semantic pointing: improving target acquisition with control-display ratio adaptation, Proceedings of the SIGCHI conference on Human factors in computing systems, p.519-526, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985758]
• 7: William Buxton , Ralph Hill , Peter Rowley, Issues and techniques in touch-sensitive tablet input, ACM SIGGRAPH Computer Graphics, v.19 n.3, p.215-224, Jul. 1985
• 8: Alan Esenther , Kathy Ryall, Fluid DTMouse: better mouse support for touch-based interactions, Proceedings of the working conference on Advanced visual interfaces, May 23-26, 2006, Venezia, Italy [doi>10.1145/1133265.1133289]
• 9: George Fitzmaurice , Azam Khan , Robert Pieké , Bill Buxton , Gordon Kurtenbach, Tracking menus, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.71-79, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964704]
• 10: Tovi Grossman , Ravin Balakrishnan, The bubble cursor: enhancing target acquisition by dynamic resizing of the cursor's activation area, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055012]
• 11: Mark S. Hancock , Kellogg S. Booth, Improving menu placement strategies for pen input, Proceedings of Graphics Interface 2004, p.221-230, May 17-19, 2004, London, Ontario, Canada
• 12: Paul Kabbash , William A. S. Buxton, The “prince” technique: Fitts' law and selection using area cursors, Proceedings of the SIGCHI conference on Human factors in computing systems, p.273-279, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223939]
• 13: Amy K. Karlson , Benjamin B. Bederson , John SanGiovanni, AppLens and launchTile: two designs for one-handed thumb use on small devices, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055001]
• 14: Karlson, A., Bederson, B., Contreras-Vidal, J. (2006) Understanding Single-Handed Mobile Device Interaction. Tech Report HCIL-2006-02.
• 15: Jennifer Mankoff , Scott E. Hudson , Gregory D. Abowd, Interaction techniques for ambiguity resolution in recognition-based interfaces, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.11-20, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354407]
• 16: Nobuyuki Matsushita , Yuji Ayatsuka , Jun Rekimoto, Dual touch: a two-handed interface for pen-based PDAs, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.211-212, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354774]
• 17: Olwal, A., Feiner S. (2003) Rubbing the Fisheye: precise touch-screen interaction with gestures and fisheye views. In Conf. Supp. of UIST'03, 83--84.
• 18: R. L. Potter , L. J. Weldon , B. Shneiderman, Improving the accuracy of touch screens: an experimental evaluation of three strategies, Proceedings of the SIGCHI conference on Human factors in computing systems, p.27-32, May 15-19, 1988, Washington, D.C., United States [doi>10.1145/57167.57171]
• 19: Xiangshi Ren , Shinju Moriya, Improving selection performance on pen-based systems: a study of pen-based interaction for selection tasks, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.3, p.384-416, Sept. 2000 [doi>10.1145/355324.355328]
• 20: IEEE Software Staff, Touch Screens Now Offer Compelling Uses, IEEE Software, v.8 n.2, p.93-94, 107, March 1991 [doi>10.1109/52.73754]
• 21: Andrew Sears , Ben Shneiderman, High precision touchscreens: design strategies and comparisons with a mouse, International Journal of Man-Machine Studies, v.34 n.4, p.593-613, April 1991 [doi>10.1016/0020-7373(91)90037-8]
• 22: Daniel Vogel , Ravin Balakrishnan, Distant freehand pointing and clicking on very large, high resolution displays, Proceedings of the 18th annual ACM symposium on User interface software and technology, October 23-26, 2005, Seattle, WA, USA [doi>10.1145/1095034.1095041]
• 23: Daniel Wigdor , Darren Leigh , Clifton Forlines , Samuel Shipman , John Barnwell , Ravin Balakrishnan , Chia Shen, Under the table interaction, Proceedings of the 19th annual ACM symposium on User interface software and technology, October 15-18, 2006, Montreux, Switzerland [doi>10.1145/1166253.1166294]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: User interfaces

  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation); Screen design (e.g., text, graphics, color)

General Terms:
Human Factors

Keywords:
interaction techniques, mobile devices, occlusion, precise target acquisition, touch-screens