HandNavigator: hands-on interaction for desktop virtual reality

ABSTRACT

This paper presents a novel interaction system, aimed at hands-on manipulation of digital models through natural hand gestures. Our system is composed of a new physical interaction device coupled with a simulated compliant virtual hand model. The physical interface consists of a SpaceNavigator, augmented with pressure sensors to detect directional forces applied by the user's fingertips. This information controls the position, orientation, and posture of the virtual hand in the same way that the SpaceNavigator uses measured forces to animate a virtual frame. In this manner, user control does not involve fatigue due to reaching gestures or holding a desired hand shape. During contact, the user has a realistic visual feedback in the form of plausible interactions between the virtual hand and its environment. Our device is well suited to any situation where hand gesture, contact, or manipulation tasks need to be performed in virtual. We demonstrate the device in several simple virtual worlds and evaluate it through a series of user studies.

REFERENCES

• 1: Boulic, R., Rezzonico, S., and Thalmann, D. 1996. Multi finger manipulation of virtual objects. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST, 67--74.
• 2: Mourad Bouzit , George Popescu , Grigore Burdea , Rares Boian, The Rutgers Master II-ND Force Feedback Glove, Proceedings of the 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, p.145, March 24-25, 2002
• 3: Chalfoun, J., Massemejean, E., Mouhamed, B., Ouesdou, F. B., and Younès, R. 2003. Anatomie de la main et relation tendon couple articulaire. In Congrès International de Modélisation Numérique Appliquée, CIMNA.
• 4: Dewaele, G., and Cani, M.-P. 2004. Virtual clay for direct hand manipulation. In Eurographics (short papers).
• 5: Dewaele, G., Devernay, F., and Horaud, R. P. 2004. Hand motion from 3d point trajectories and a smooth surface model. In Proceedings of the 8th European Conference on Computer Vision, Springer, T. Pajdla and J. Matas, Eds., vol. I of LNCS 3021, 495--507.
• 6: Lionel Dominjon , Anatole Lecuyer , Jean-Marie Burkhardt , Guillermo Andrade-Barroso , Simon Richir, The "Bubble" Technique: Interacting with Large Virtual Environments Using Haptic Devices with Limited Workspace, Proceedings of the First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, p.639-640, March 18-20, 2005 [doi>10.1109/WHC.2005.126]
• 7: Lionel Dominjon , Simon Richir , Anatole Lecuyer , Jean-Marie Burkhardt, Haptic Hybrid Rotations: Overcoming Hardware Angular Limitations of Force-Feedback Devices, Proceedings of the IEEE conference on Virtual Reality, p.164-174, March 25-29, 2006 [doi>10.1109/VR.2006.68]
• 8: Hillebrand, G., Bauer, M., Achatz, K., and Klinker, G. 2006. Inverse kinematic infrared optical finger tracking. In Proceedings of 9th International Conference on Humans and Computers (HC 2006), Aizu, Japan.
• 9: Brent E Insko , Michael J Meehan , Mary C Whitton , Frederick P Brooks, Jr., Passive Haptics Significantly Enhances Virtual Environments, University of North Carolina at Chapel Hill, Chapel Hill, NC, 2001
• 10: Johansson, R. S. 1998. Sensory input and control of grip. In Sensory Guidance of Movement. Novartis Foundation Symposium 218, Wiley & Sons, Chichester, 45--59.
• 11: Seahak Kim , Masahiro Ishii , Yasuharu Koike , Makoto Sato, Development of tension based haptic interface and possibility of its application to virtual reality, Proceedings of the ACM symposium on Virtual reality software and technology, October 22-25, 2000, Seoul, Korea [doi>10.1145/502390.502428]
• 12: Kry, P. G., and Pai, D. K. 2006. Grasp recognition and manipulation with the tango. In International Symposium on Experimental Robotics, Rio de Janeiro, Brazil, July 6--10, 9 pages.
• 13: Paul G. Kry , Dinesh K. Pai, Interaction capture and synthesis, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
• 14: Gregorij Kurillo , Matjaž Mihelj , Marko Munih , Tadej Bajd, Multi-Fingered Grasping and Manipulation in Virtual Environments Using an Isometric Finger Device, Presence: Teleoperators and Virtual Environments, v.16 n.3, p.293-306, June 2007 [doi>10.1162/pres.16.3.293]
• 15: Anatole Lecuyer , Sabine Coquillart , Abderrahmane Kheddar , Paul Richard , Philippe Coiffet, Pseudo-Haptic Feedback: Can Isometric Input Devices Simulate Force Feedback?, Proceedings of the IEEE Virtual Reality 2000 Conference, p.83, March 18-22, 2000
• 16: Dinesh K. Pai , Eric W. VanDerLoo , Subarna Sadhukhan, The Tango: A Tangible Tangoreceptive Whole-Hand Human Interface, Proceedings of the First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, p.141-147, March 18-20, 2005 [doi>10.1109/WHC.2005.136]
• 17: Pihuit, A., Kry, P., and Cani, M.-P. 2008. Hands-on virtual clay. In IEEE International Conference on Shape Modeling and Applications (SMI), G. X. Spagnuolo M., Cohen-Or D., Ed., Shape Modeling Interface (SMI), 267--268.
• 18: Nancy S. Pollard , Victor Brian Zordan, Physically based grasping control from example, Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 29-31, 2005, Los Angeles, California [doi>10.1145/1073368.1073413]
• 19: Santello, M., Flanders, M., and Soechting, J. 1998. Postural hand synergies for tool use. In The Journal of Neuroscience.
• 20: Markus Schlattman , Reinhard Klein, Simultaneous 4 gestures 6 DOF real-time two-hand tracking without any markers, Proceedings of the 2007 ACM symposium on Virtual reality software and technology, November 05-07, 2007, Newport Beach, California [doi>10.1145/1315184.1315188]
• 21: Jia Sheng , Ravin Balakrishnan , Karan Singh, An interface for virtual 3D sculpting via physical proxy, Proceedings of the 4th international conference on Computer graphics and interactive techniques in Australasia and Southeast Asia, November 29-December 02, 2006, Kuala Lumpur, Malaysia [doi>10.1145/1174429.1174467]
• 22: D. J. Sturman , D. Zeltzer , S. Pieper, Hands-on interaction with virtual environments, Proceedings of the 2nd annual ACM SIGGRAPH symposium on User interface software and technology, p.19-24, November 13-15, 1989, Williamsburg, Virginia, United States [doi>10.1145/73660.73663]
• 23: Winnie Tsang , Karan Singh , Eugene Fiume, Helping hand: an anatomically accurate inverse dynamics solution for unconstrained hand motion, Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 29-31, 2005, Los Angeles, California [doi>10.1145/1073368.1073414]
• 24: Shumin Zhai , Paul Milgram , William Buxton, The influence of muscle groups on performance of multiple degree-of-freedom input, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.308-315, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238534]
• 25: Zhai, S. 1995. Human Performance in Six Degree of Freedom Input Control. PhD thesis, University of Toronto.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Virtual reality

Keywords:
hands, interaction, virtual reality