Detecting and leveraging finger orientation for interaction with direct-touch surfaces

ABSTRACT

Current interactions on direct-touch interactive surfaces are often modeled based on properties of the input channel that are common in traditional graphical user interfaces (GUI) such as x-y coordinate information. Leveraging additional information available on the surfaces could potentially result in richer and novel interactions. In this paper we specifically explore the role of finger orientation. This property is typically ignored in touch-based interactions partly because of the ambiguity in determining it solely from the contact shape. We present a simple algorithm that unambiguously detects the directed finger orientation vector in real-time from contact information only, by considering the dynamics of the finger landing process. Results of an experimental evaluation show that our algorithm is stable and accurate. We then demonstrate how finger orientation can be leveraged to enable novel interactions and to infer higher-level information such as hand occlusion or user position. We present a set of orientation-aware interaction techniques and widgets for direct-touch surfaces.

REFERENCES

• 1: Albinsson, P.r.-A. and Zhai, S., High precision touch screen interaction. CHI 2003, 105--112.
• 2: Benko, H., Wilson, A.D. and Balakrishnan, R., Sphere: multi-touch interactions on a spherical display. UIST 2008, 77--86.
• 3: Benko, H., Wilson, A.D. and Baudisch, P., Precise selection techniques for multi-touch screens. CHI 2006, 1263001272.
• 4: Blanch, R., Guiard, Y. and Beaudouin-Lafon, M., Semantic pointing: improving target acquisition with control-display ratio adaptation. CHI 2004, 519--526.
• 5: Blasko, G., Beaver, W., Kamvar, M., and Feiner, S., Workplane-orientation-sensing techniques for tablet PCs. UIST 2004 Conf. Supplement, 1--2.
• 6: Brandl, P., Seifried, T., Leitner, J., Haller, M., Doray, B. and To, P., Occlusion-Aware Menu Design for Digital Tabletops. CHI 2009, 3223--3228.
• 7: Cao, X., Wilson, A. D., Balakrishnan, R., Hinckley, K. and Hudson, S., ShapeTouch: Leveraging contact shape on interactive surfaces. TABLETOP 2008, 129--136
• 8: Davidson, P. L. and Han, J. Y., Extending 2D object arrangement with pressure-sensitive layering cues. UIST 2008, 87--90.
• 9: Dietz, P. and Leigh, D., DiamondTouch: a multi-user touch technology. UIST 2001, 219--226.
• 10: Esenther, A., Forlines, C., Ryall, K. and Shipman, S. Support for Multi-User, Multi-Touch Applications, 2002.
• 11: Esenther, A. and Ryall, K., Fluid DTMouse: better mouse support for touch-based interactions. AVI 2006, 112--115.
• 12: Forlines, C., Wigdor, D., Shen, C. and Balakrishnan, R., Direct-touch vs. mouse input for tabletop displays. CHI 2007, 647--656.
• 13: Grossman, T. and Balakrishnan, R., The bubble cursor: enhancing target acquisition by dynamic resizing of the cursor's activation area. CHI 2005, 281--290.
• 14: Grossman, T. and Balakrishnan, R., The design and evaluation of selection techniques for 3D volumetric displays. UIST 2006, 3--12.
• 15: Han, J. Y., Low-cost multi-touch sensing through frustrated total internal reflection. UIST 2005, 115--118.
• 16: Izadi, S., Hodges, S., Butler, A., Rrustemi, A. and Buxton, B., ThinSight: integrated optical multi-touch sensing through thin form-factor displays. workshop on Emerging displays technologies, 6.
• 17: Kabbash, P. and Buxton, W.A.S., The "prince" technique: Fitts' law and selection using area cursors. CHI 1995, 273--279.
• 18: Lee, S., Buxton, W. and Smith, K.C., A multi-touch three dimensional touch-sensitive tablet. CHI 1985, ACM, 21--25.
• 19: Malik, S. and Laszlo, J., Visual touchpad: a two-handed gestural input device. Multimodal interfaces 2004, 289--296.
• 20: Matejka, J., Grossman, T., Lo, J. and Fitzmaurice, G., The Design and Evaluation of Multi-Finger Mouse Emulation Techniques. CHI 2009, 1073--1082.
• 21: Microsoft. Microsoft Surface, 2007, http://www.microsoft.com/surface.
• 22: Morris, M.J. Supporting Effective Interaction With Tabletop Groupware, Meredith June Morris, Stanford University, 2006.
• 23: Moscovich, T. Principles and Applications of Multi-touch Interaction, Brown University, 2007.
• 24: Moscovich, T. and Hughes, J.F., Indirect mappings of multi-touch input using one and two hands. CHI 2008, 1275--1284.
• 25: Moscovich, T. and Hughes, J. F., Multi-finger cursor techniques. GI 2006, 1--7.
• 26: Nuigroup. Touchlib, Nui Group, 2009. http://www.nuigroup.com
• 27: Olwal, A. and Feiner, S., Rubbing the Fisheye: precise touch-screen interaction with gestures and fisheye views. UIST 2003 Supplyment, 83--84.
• 28: Potter, R.L., Weldon, L.J. and Shneiderman, B., Improving the accuracy of touch screens: an experimental evaluation of three strategies. CHI 1988, 27--32.
• 29: Rekimoto, J., SmartSkin: an infrastructure for freehand manipulation on interactive surfaces. CHI 2002, 113--120.
• 30: Scott, S. D., Carpendale, M. S. T. and Inkpen, K. M., Territoriality in Collaborative Tabletop Workspaces. CSCW 2004, ACM, 294--303.
• 31: Tian, F., Xu, L., Wang, H., Zhang, X., Liu, Y., Setlur, V. and Dai, G., Tilt menu: using the 3D orientation information of pen devices to extend the selection capability of pen-based user interfaces. CHI 2008, 1371--1380.
• 32: Vogel, D. and Baudisch, P., Shift: a technique for operating pen-based interfaces using touch. CHI 2007, 657--666.
• 33: Wang, F. and Ren, X., Empirical Evaluation for Finger Input Properties In Multi-touch Interaction. CHI 2009, 1063--1072.
• 34: Wilson, A.D. PlayAnywhere: a compact interactive tabletop projection-vision system. UIST 2005, 83--92.
• 35: Wilson, A.D., TouchLight: an imaging touch screen and display for gesture-based interaction. Multimodal interfaces 2004, 69--76.
• 36: Wilson, A.D., Izadi, S., Hilliges, O., Garcia-Mendoza, A. and Kirk, D., Bringing physics to the surface. UIST 2008, 67--76.
• 37: Wu, M. and Balakrishnan, R., Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays. UIST 2003, 193--202.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.6 Methodology and Techniques

General Terms:
Design, Human Factors

Keywords:
direct-touch surface, finger orientation, orientation aware interface