Using an event-based approach to improve the multimodal rendering of 6DOF virtual contact

ABSTRACT

This paper decribes a general event-based approach to improve multimodal rendering of 6DOF (degree of freedom) contact between objects in interactive virtual object simulations. The contact events represent the different steps of two objects colliding with each other: (1) the state of free motion, (2) the impact event at the moment of collision (3) the friction state during the contact and (4) the detachment event at the end of the contact. The different events are used to improve the classical feedback by superimposing specific rendering techniques based on these events. First we propose a general method to generate these events based only on the objects' positions given by the simulation. Second, we describe a set of different types of multimodal feedback associated to the different events that we implemented in a complex virtual simulation dedicated to virtual assembly. For instance, we propose a visual rendering of impact, friction and detachment based on particle effects. We used the impact event to improve the 6DOF haptic rendering by superimposing a high frequency force pattern to the classical force feedback. We also implemented a realistic audio rendering using impact and friction sound on the corresponding events. All these first implementations can be easily extended with other event-based effects on various rigid body simulations thanks to our modular approach.

REFERENCES

• 1: Federico Avanzini , Paolo Crosato, Integrating physically based sound models in a multimodal rendering architecture: Research Articles, Computer Animation and Virtual Worlds, v.17 n.3-4, p.411-419, July 2006 [doi>10.1002/cav.v17:3/4]
• 2: Boff, K. R., and Lincoln, J. E. 1988. Engineering data compendium: Human perception and performance. Tech. Rep. 3, Armstrong Aerospace Research Laboratory, Wright-Patterson AFB.
• 3: Cadoz, C., annie Luciani, and Florens, J.-L. 1984. Responsive input devices and sound synthesis by simulation of instrumental mechanisms: The cordis system. Computer Music Journal 8, 3, 60--73.
• 4: Colgate, J. E., Grafing, P. E., Stanley, M. C., and Schenkel, G. 1993. Implementation of stiff virtual walls in force-reflecting interfaces. 202--208.
• 5: Margaret Diane Rezvan R. Minsky, Computational haptics: the sandpaper system for synthesizing texture for a force-feedback display, Massachusetts Institute of Technology, Cambridge, MA, 1996
• 6: Derek DiFilippo , Dinesh K. Pai, The AHI: an audio and haptic interface for contact interactions, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.149-158, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354437]
• 7: Nicola Diolaiti , Gunter Niemeyer , Federico Barbagli , J. Kenneth Salisbury , Claudio Melchiorri, The Effect of Quantization and Coulomb Friction on the Stability of Haptic Rendering, Proceedings of the First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, p.237-246, March 18-20, 2005 [doi>10.1109/WHC.2005.130]
• 8: Gomes de Sá, A., and Zachmann, G. 1999. Virtual reality as a tool for verification of assembly and maintenance processes. Computers and Graphics 23, 3, 389--403.
• 9: Haption, 2003. Haption Virtuose 6D35-45 technical specifications. http://www.haption.com.
• 10: Helen H. Hu , Amy A. Gooch , William B. Thompson , Brian E. Smits , John J. Rieser , Peter Shirley, Visual cues for imminent object contact in realistic virtual environment, Proceedings of the conference on Visualization '00, p.179-185, October 2000, Salt Lake City, Utah, United States
• 11: Hwang, J. D., Williams, M. D., and Niemeyer, G. 2004. Toward event-based haptics: Rendering contact using open-loop force pulses. In Proceedings of the conference on Haptic Interfaces for Virtual Environments and Teleoperator Systems, 24--31.
• 12: Janabi-Sharifi, F., Hayward, V., and Chen, C.-S. J. 2000. Discrete-time adaptive windowing for velocity estimation. IEEE Transactions on Control Systems technology 8, 6, 1003+.
• 13: Jimnez, P., Thomas, F., and Torras, C. 2001. 3D Collision Detection: A Survey. Computers and Graphics 25, 2, 269--285.
• 14: David E. Johnson , Peter Willemsen, Six Degree-of-Freedom Haptic Rendering of Complex Polygonal Models, Proceedings of the 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS'03), p.229, March 22-23, 2003
• 15: Katherine J. Kuchenbecker , Jonathan Fiene , Gunter Niemeyer, Improving Contact Realism through Event-Based Haptic Feedback, IEEE Transactions on Visualization and Computer Graphics, v.12 n.2, p.219-230, March 2006 [doi>10.1109/TVCG.2006.32]
• 16: Lécuyer, A., Mégard, C., Burkhardt, J.-M., Lim, T., Coquillart, S., Coiffet, P., and Graux, L. 2002. The effect of haptic, visual and auditory feedback on an insertion task on a 2-screen workbench. In Proceedings of the Immersive Projection Technology Symposium.
• 17: William A. McNeely , Kevin D. Puterbaugh , James J. Troy, Six degree-of-freedom haptic rendering using voxel sampling, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.401-408, July 1999 [doi>10.1145/311535.311600]
• 18: Merlhiot, X. 2005. GVM/LMD++ physics engine. Tech. Rep. DTSI/SCRI/LCI/05RT.011, CEA.
• 19: Niemeyer, G., and Slotine, J.-J. E. 1991. Stable adaptive teleoperation. IEEE Journal of Oceanic Engineering 16, 1, 152--162.
• 20: James F. O'Brien , Perry R. Cook , Georg Essl, Synthesizing sounds from physically based motion, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.529-536, August 2001 [doi>10.1145/383259.383321]
• 21: Michael Ortega , Stephane Redon , Sabine Coquillart, A Six Degree-of-Freedom God-Object Method for Haptic Display of Rigid Bodies with Surface Properties, IEEE Transactions on Visualization and Computer Graphics, v.13 n.3, p.458-469, May 2007 [doi>10.1109/TVCG.2007.1028]
• 22: Miguel A. Otaduy , Nitin Jain , Avneesh Sud , Ming C. Lin, Haptic Display of Interaction between Textured Models, Proceedings of the conference on Visualization '04, p.297-304, October 10-15, 2004 [doi>10.1109/VIS.2004.37]
• 23: Pocheville, A., and Kheddar, A. 2004. I-TOUCH: a framework for computer haptics. In Proceedings of the International Conference on Intelligent Robots and System.
• 24: Pulkki, V. 1997. Virtual sound source positioning using vector base amplitude. Journal of the Audio Engineering. Society 45, 6, 456--466.
• 25: Raghuvanshi, N., and Lin, M. C. 2006. Symphony: Realtime physically-based sound synthesis. In Proceedings of Symposium on Interactive 3D Graphics and Games.
• 26: Redon, S., Kheddar, A., and Coquillard, S. 2000. An algebraic solution to the problem of collision detection for rigid polyhedral objects. In Proceedings of the International Conference on Robotics and Automation, 3733--3738.
• 27: Ruspini, D., and Khatib, O. 2000. A framework for multi-contact multi-body dynamic simulation and haptic display. In Proceedings of International Conference on Intelligent Robots and Systems, 1322--1327.
• 28: Siira, J., and Pai, D. K. 1996. Haptic texturing - a stochastic approach. In Proceedings of the International Conference on Robotics and Automation, 557--562.
• 29: Jean Sreng , Anatole Lecuyer , Christine Megard , Claude Andriot, Using Visual Cues of Contact to Improve Interactive Manipulation of Virtual Objects in Industrial Assembly/Maintenance Simulations, IEEE Transactions on Visualization and Computer Graphics, v.12 n.5, p.1013-1020, September 2006 [doi>10.1109/TVCG.2006.189]
• 30: Stefano Stramigioli, Modeling and IPC Control of Interactive Mechanical Systems: A Coordinate-Free Approach, Springer-Verlag New York, Inc., Secaucus, NJ, 2001
• 31: Tapio Takala , James Hahn, Sound rendering, ACM SIGGRAPH Computer Graphics, v.26 n.2, p.211-220, July 1992
• 32: Kees van den Doel , Paul G. Kry , Dinesh K. Pai, FoleyAutomatic: physically-based sound effects for interactive simulation and animation, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.537-544, August 2001 [doi>10.1145/383259.383322]
• 33: Warren, W. H., and Verbrugge, R. R. 1984. Auditory perception of breaking and bouncing events: a case study in ecological acoustics. Journal of experimental psychology. Human perception and performance 10, 5, 704--712.
• 34: Yano, H., Igawa, H., Kameda, T., Muzutani, K., and Iwata, H. 2004. Audiohaptics: audio and haptic rendering based on a physical model. In Proceedings of Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 250--257.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Artificial, augmented, and virtual realities

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Haptic I/O; Auditory (non-speech) feedback

Keywords:
6DOF, audio, contact, event-based, haptic, multimodal, rendering, visual