Bringing physics to the surface

ABSTRACT

This paper explores the intersection of emerging surface technologies, capable of sensing multiple contacts and of-ten shape information, and advanced games physics engines. We define a technique for modeling the data sensed from such surfaces as input within a physics simulation. This affords the user the ability to interact with digital objects in ways analogous to manipulation of real objects. Our technique is capable of modeling both multiple contact points and more sophisticated shape information, such as the entire hand or other physical objects, and of mapping this user input to contact forces due to friction and collisions within the physics simulation. This enables a variety of fine-grained and casual interactions, supporting finger-based, whole-hand, and tangible input. We demonstrate how our technique can be used to add real-world dynamics to interactive surfaces such as a vision-based tabletop, creating a fluid and natural experience. Our approach hides from application developers many of the complexities inherent in using physics engines, allowing the creation of applications without preprogrammed interaction behavior or gesture recognition.

REFERENCES

• 1: Anand Agarawala , Ravin Balakrishnan, Keepin' it real: pushing the desktop metaphor with physics, piles and the pen, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124965]
• 2: Barron, J., Fleet, D., Beauchemin, S., and Burkitt, T. 1992. Performance of optical flow techniques. Computer Vision and Pattern Recognition, 236--242.
• 3: D. Baraff, Analytical methods for dynamic simulation of non-penetrating rigid bodies, ACM SIGGRAPH Computer Graphics, v.23 n.3, p.223-232, July 1989
• 4: David Baraff, Fast contact force computation for nonpenetrating rigid bodies, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.23-34, July 1994 [doi>10.1145/192161.192168]
• 5: Bay-Wei Chang , David Ungar, Animation: from cartoons to the user interface, Proceedings of the 6th annual ACM symposium on User interface software and technology, p.45-55, December 1993, Atlanta, Georgia, United States [doi>10.1145/168642.168647]
• 6: Fu Chang , Chun-Jen Chen , Chi-Jen Lu, A linear-time component-labeling algorithm using contour tracing technique, Computer Vision and Image Understanding, v.93 n.2, p.206-220, February 2004 [doi>10.1016/j.cviu.2003.09.002]
• 7: Paul Dietz , Darren Leigh, DiamondTouch: a multi-user touch technology, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502389]
• 8: Pierre Dragicevic, Combining crossing-based and paper-based interaction paradigms for dragging and dropping between overlapping windows, Proceedings of the 17th annual ACM symposium on User interface software and technology, October 24-27, 2004, Santa Fe, NM, USA [doi>10.1145/1029632.1029667]
• 9: Kenny Erleben , Jon Sporring , Knud Henriksen , Kenrik Dohlman, Physics-based Animation (Graphics Series), Charles River Media, Inc., Rockland, MA, 2005
• 10: Bernd Fröhlich , Henrik Tramberend , Andrew Beers , Maneesh Agrawala , David Baraff, Physically-Based Manipulation on the Responsive Workbench, Proceedings of the IEEE Virtual Reality 2000 Conference, p.5, March 18-22, 2000
• 11: Rafael C. Gonzalez , Richard E. Woods, Digital Image Processing (3rd Edition), Prentice-Hall, Inc., Upper Saddle River, NJ, 2006
• 12: Jörg Geißler, Shuffle, throw or take it! working efficiently with an interactive wall, CHI 98 conference summary on Human factors in computing systems, p.265-266, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/286498.286745]
• 13: Grossman, T., and Wigdor, D. 2007. Going deeper: a taxonomy of 3D on the tabletop. Second IEEE International Workshop on Horizontal Interactive Human-Computer Systems, 137--144.
• 14: Jefferson Y. Han, Low-cost multi-touch sensing through frustrated total internal reflection, Proceedings of the 18th annual ACM symposium on User interface software and technology, October 23-26, 2005, Seattle, WA, USA [doi>10.1145/1095034.1095054]
• 15: Mark Hancock , Sheelagh Carpendale , Andy Cockburn, Shallow-depth 3d interaction: design and evaluation of one-, two- and three-touch techniques, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240798]
• 16: Hancock, M., Carpendale, S., Supporting Multiple Off-Axis Viewpoints at a Tabletop Display. 2007. Second IEEE International Workshop on Horizontal Interactive Human-Computer Systems, 171--178.
• 17: Russell Kruger , Sheelagh Carpendale , Stacey D. Scott , Anthony Tang, Fluid integration of rotation and translation, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055055]
• 18: Jun Liu , David Pinelle , Samer Sallam , Sriram Subramanian , Carl Gutwin, TNT: improved rotation and translation on digital tables, Proceedings of Graphics Interface 2006, June 07-09, 2006, Quebec, Canada
• 19: Richard Mander , Gitta Salomon , Yin Yin Wong, A “pile” metaphor for supporting casual organization of information, Proceedings of the SIGCHI conference on Human factors in computing systems, p.627-634, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.143055]
• 20: Microsoft Corporation. Microsoft Surface. http://ww.surface.com. 2007.
• 21: NVIDIA Corporation. NVIDIA PhysX. http://www.nvidia.com/object/nvidia_physx.html. 2008.
• 22: Adrian Reetz , Carl Gutwin , Tadeusz Stach , Miguel Nacenta , Sriram Subramanian, Superflick: a natural and efficient technique for long-distance object placement on digital tables, Proceedings of Graphics Interface 2006, June 07-09, 2006, Quebec, Canada
• 23: George Robertson , Mary Czerwinski , Kevin Larson , Daniel C. Robbins , David Thiel , Maarten van Dantzich, Data mountain: using spatial memory for document management, Proceedings of the 11th annual ACM symposium on User interface software and technology, p.153-162, November 01-04, 1998, San Francisco, California, United States [doi>10.1145/288392.288596]
• 24: Olov Ståhl , Anders Wallberg , Jonas Söderberg , Jan Humble , Lennart E. Fahlén , Adrian Bullock , Jenny Lundberg, Information exploration using The Pond, Proceedings of the 4th international conference on Collaborative virtual environments, p.72-79, September 30-October 02, 2002, Bonn, Germany [doi>10.1145/571878.571890]
• 25: Andrew D. Wilson, PlayAnywhere: a compact interactive tabletop projection-vision system, Proceedings of the 18th annual ACM symposium on User interface software and technology, October 23-26, 2005, Seattle, WA, USA [doi>10.1145/1095034.1095047]
• 26: Wilson, A. 2007. Depth-sensing video cameras for 3D Tangible Interaction. Second IEEE International Workshop on Horizontal Interactive Human-Computer Systems, 201--204.
• 27: Mike Wu , Ravin Balakrishnan, Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.193-202, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964718]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

General Terms:
Algorithms, Design, Human Factors

Keywords:
game physics engines, interactive surfaces