Robust computer vision-based detection of pinching for one and two-handed gesture input

ABSTRACT

We present a computer vision technique to detect when the user brings their thumb and forefinger together (a pinch gesture) for close-range and relatively controlled viewing circumstances. The technique avoids complex and fragile hand tracking algorithms by detecting the hole formed when the thumb and forefinger are touching; this hole is found by simple analysis of the connected components of the background segmented against the hand. Our Thumb and Fore-Finger Interface (TAFFI) demonstrates the technique for cursor control as well as map navigation using one and two-handed interactions.

REFERENCES

• 1: Ravin Balakrishnan , I. Scott MacKenzie, Performance differences in the fingers, wrist, and forearm in computer input control, Proceedings of the SIGCHI conference on Human factors in computing systems, p.303-310, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258764]
• 2: Thomas Baudel , Michel Beaudouin-Lafon, Charade: remote control of objects using free-hand gestures, Communications of the ACM, v.36 n.7, p.28-35, July 1993 [doi>10.1145/159544.159562]
• 3: Bowman, D., Wingrave, C., Campbell, J., Ly, V. and Rhoton, C. Novel uses of pinch gloves for virtual environment interaction techniques. Virtual Reality, 6 (3). 122--129.
• 4: William Buxton, A three-state model of graphical input, Proceedings of the IFIP TC13 Third Interational Conference on Human-Computer Interaction, p.449-456, August 27-31, 1990
• 5: Ken Hinckley , Mary Czerwinski , Mike Sinclair, Interaction and modeling techniques for desktop two-handed input, Proceedings of the 11th annual ACM symposium on User interface software and technology, p.49-58, November 01-04, 1998, San Francisco, California, United States [doi>10.1145/288392.288572]
• 6: Berthold K. Horn, Robot Vision, McGraw-Hill Higher Education, 1986
• 7: Krueger, M. W. Artificial Reality II. Addison-Wesley, Menlo Park, 1991.
• 8: Gordon Kurtenbach , George Fitzmaurice , Thomas Baudel , Bill Buxton, The design of a GUI paradigm based on tablets, two-hands, and transparency, Proceedings of the SIGCHI conference on Human factors in computing systems, p.35-42, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258574]
• 9: I. Scott MacKenzie , Aleks Oniszczak, A comparison of three selection techniques for touchpads, Proceedings of the SIGCHI conference on Human factors in computing systems, p.336-343, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274691]
• 10: Shahzad Malik , Joe Laszlo, Visual touchpad: a two-handed gestural input device, Proceedings of the 6th international conference on Multimodal interfaces, October 13-15, 2004, State College, PA, USA [doi>10.1145/1027933.1027980]
• 11: Quek, F., Mysliwiec, T. and Zhao, M., FingerMouse: A freehand pointing interface. in IEEE Automatic Face and Gesture Recognition, (1995), 372--377.
• 12: Andrew D. Wilson, PlayAnywhere: a compact interactive tabletop projection-vision system, Proceedings of the 18th annual ACM symposium on User interface software and technology, October 23-26, 2005, Seattle, WA, USA [doi>10.1145/1095034.1095047]
• 13: Wilson, A. and Cutrell, E., FlowMouse: A computer vision-based pointing and gesture input device. in INTERACT '05, (Rome, Italy, 2005).

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

Additional Classification:
  B. Hardware
  B.4 INPUT/OUTPUT AND DATA COMMUNICATIONS
      B.4.2 Input/Output Devices

Keywords:
bimanual interaction, computer vision, gesture, hand tracking, navigation