Tilt menu: using the 3D orientation information of pen devices to extend the selection capability of pen-based user interfaces

ABSTRACT

We present a new technique called 'Tilt Menu' for better extending selection capabilities of pen-based interfaces. The Tilt Menu is implemented by using 3D orientation information of pen devices while performing selection tasks. The Tilt Menu has the potential to aid traditional one-handed techniques as it simultaneously generates the secondary input (e.g., a command or parameter selection) while drawing/interacting with a pen tip without having to use the second hand or another device. We conduct two experiments to explore the performance of the Tilt Menu. In the first experiment, we analyze the effect of parameters of the Tilt Menu, such as the menu size and orientation of the item, on its usability. Results of the first experiment suggest some design guidelines for the Tilt Menu. In the second experiment, the Tilt Menu is compared to two types of techniques while performing connect-the-dot tasks using freeform drawing mechanism. Results of the second experiment show that the Tilt Menu perform better in comparison to the Tool Palette, and is as good as the Toolglass.

REFERENCES

• 1: Adobe Illustrator, http://www.adobe.com/products/illustrator/
• 2: Johnny Accot , Shumin Zhai, Beyond Fitts' law: models for trajectory-based HCI tasks, Proceedings of the SIGCHI conference on Human factors in computing systems, p.295-302, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258760]
• 3: James Arvo , Kevin Novins, Fluid sketches: continuous recognition and morphing of simple hand-drawn shapes, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.73-80, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354413]
• 4: Ravin Balakrishnan , Thomas Baudel , Gordon Kurtenbach , George Fitzmaurice, The Rockin'Mouse: integral 3D manipulation on a plane, Proceedings of the SIGCHI conference on Human factors in computing systems, p.311-318, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258778]
• 5: Ravin Balakrishnan , Ken Hinckley, The role of kinesthetic reference frames in two-handed input performance, Proceedings of the 12th annual ACM symposium on User interface software and technology, p.171-178, November 07-10, 1999, Asheville, North Carolina, United States [doi>10.1145/320719.322599]
• 6: Eric A. Bier , Maureen C. Stone , Ken Pier , William Buxton , Tony D. DeRose, Toolglass and magic lenses: the see-through interface, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.73-80, September 1993 [doi>10.1145/166117.166126]
• 7: J. Callahan , D. Hopkins , M. Weiser , B. Shneiderman, An empirical comparison of pie vs. linear menus, Proceedings of the SIGCHI conference on Human factors in computing systems, p.95-100, May 15-19, 1988, Washington, D.C., United States [doi>10.1145/57167.57182]
• 8: George Fitzmaurice , Azam Khan , Robert Pieké , Bill Buxton , Gordon Kurtenbach, Tracking menus, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.71-79, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964704]
• 9: Franke, K., The influence of physical and biomechanical processes on the ink trace methodological foundations for the forensic analysis of signatures, Ph.D. thesis, University of Groningen, The Netherlands, 2005.
• 10: Tovi Grossman , Ken Hinckley , Patrick Baudisch , Maneesh Agrawala , Ravin Balakrishnan, Hover widgets: using the tracking state to extend the capabilities of pen-operated devices, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124898]
• 11: François Guimbretiére , Terry Winograd, FlowMenu: combining command, text, and data entry, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.213-216, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354778]
• 12: François Guimbretière , Andrew Martin , Terry Winograd, Benefits of merging command selection and direct manipulation, ACM Transactions on Computer-Human Interaction (TOCHI), v.12 n.3, p.460-476, September 2005 [doi>10.1145/1096737.1096742]
• 13: Ken Hinckley , Francois Guimbretiere , Maneesh Agrawala , Georg Apitz , Nicholas Chen, Phrasing techniques for multi-stroke selection gestures, Proceedings of Graphics Interface 2006, June 07-09, 2006, Quebec, Canada
• 14: Robert J. K. Jacob , Leonidas Deligiannidis , Stephen Morrison, A software model and specification language for non-WIMP user interfaces, ACM Transactions on Computer-Human Interaction (TOCHI), v.6 n.1, p.1-46, March 1999 [doi>10.1145/310641.310642]
• 15: V. Babu , L. Prasanth , R. Sharma , G. V. Rao , A. Bharath, HMM-Based Online Handwriting Recognition System for Telugu Symbols, Proceedings of the Ninth International Conference on Document Analysis and Recognition (ICDAR 2007) Vol 1, p.63-67, September 23-26, 2007
• 16: Paul Kabbash , William Buxton , Abigail Sellen, Two-handed input in a compound task, Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, p.417-423, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/191666.191808]
• 17: Gordon Kurtenbach , William Buxton, The limits of expert performance using hierarchic marking menus, Proceedings of the INTERACT '93 and CHI '93 conference on Human factors in computing systems, p.482-487, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169426]
• 18: Kurtenbach, G., Sellen, A., Buxton, W., An empirical evaluation of some articulatory and cognitive aspects of "marking menus". Human Computer Interaction, 1993. 8(1): 1--23.
• 19: Junfeng Li , Xiwen Zhang , Xiang Ao , Guozhong Dai, Sketch recognition with continuous feedback based on incremental intention extraction, Proceedings of the 10th international conference on Intelligent user interfaces, January 10-13, 2005, San Diego, California, USA [doi>10.1145/1040830.1040866]
• 20: Yang Li , Ken Hinckley , Zhiwei Guan , James A. Landay, Experimental analysis of mode switching techniques in pen-based user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055036]
• 21: Daniel L. Odell , Richard C. Davis , Andrew Smith , Paul K. Wright, Toolglasses, marking menus, and hotkeys: a comparison of one and two-handed command selection techniques, Proceedings of Graphics Interface 2004, p.17-24, May 17-19, 2004, London, Ontario, Canada
• 22: Kurt Partridge , Saurav Chatterjee , Vibha Sazawal , Gaetano Borriello , Roy Want, TiltType: accelerometer-supported text entry for very small devices, Proceedings of the 15th annual ACM symposium on User interface software and technology, October 27-30, 2002, Paris, France [doi>10.1145/571985.572013]
• 23: Stuart Pook , Eric Lecolinet , Guy Vaysseix , Emmanuel Barillot, Control menus: excecution and control in a single interactor, CHI '00 extended abstracts on Human factors in computing systems, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/633292.633446]
• 24: Solidworks, http://www.solidworks.com
• 25: Gonzalo Ramos , Matthew Boulos , Ravin Balakrishnan, Pressure widgets, Proceedings of the SIGCHI conference on Human factors in computing systems, p.487-494, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985754]
• 26: Gonzalo Ramos , Ravin Balakrishnan, Zliding: fluid zooming and sliding for high precision parameter manipulation, Proceedings of the 18th annual ACM symposium on User interface software and technology, October 23-26, 2005, Seattle, WA, USA [doi>10.1145/1095034.1095059]
• 27: Gonzalo A. Ramos , Ravin Balakrishnan, Pressure marks, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240834]
• 28: Eric Saund , Edward Lank, Stylus input and editing without prior selection of mode, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.213-216, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964720]
• 29: Tandler, P., Prante, T., Using Incremental Gesture Recognition to Provide Immediate Feedback while Drawing Pen Gestures. In Proc. UIST 2001, ACM Press (2001), 18--25.
• 30: Feng Tian , Xiang Ao , Hongan Wang , Vidya Setlur , Guozhong Dai, The tilt cursor: enhancing stimulus-response compatibility by providing 3d orientation cue of pen, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240675]
• 31: Daniel Wigdor , Ravin Balakrishnan, TiltText: using tilt for text input to mobile phones, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.81-90, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964705]
• 32: Shengdong Zhao , Maneesh Agrawala , Ken Hinckley, Zone and polygon menus: using relative position to increase the breadth of multi-stroke marking menus, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124933]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation); Theory and methods

General Terms:
Design, Human Factors

Keywords:
orientation, pen, pen-based ui, selection, tilt menu