PhotoelasticTouch: transparent rubbery tangible interface using an LCD and photoelasticity

ABSTRACT

PhotoelasticTouch is a novel tabletop system designed to intuitively facilitate touch-based interaction via real objects made from transparent elastic material. The system utilizes vision-based recognition techniques and the photoelastic properties of the transparent rubber to recognize deformed regions of the elastic material. Our system works with elastic materials over a wide variety of shapes and does not require any explicit visual markers. Compared to traditional interactive surfaces, our 2.5 dimensional interface system enables direct touch interaction and soft tactile feedback. In this paper we present our force sensing technique using photoelasticity and describe the implementation of our prototype system. We also present three practical applications of PhotoelasticTouch, a force-sensitive touch panel, a tangible face application, and a paint application.

REFERENCES

• 1: Microsoft Surface. http://www.microsoft.com/surface/.
• 2: Paul Dietz and Darren Leigh. Diamondtouch: a multiuser touch technology. In UIST '01: Proceedings of the 14th annual ACM symposium on User interface software and technology, pages 219--226, New York, NY, USA, 2001. ACM.
• 3: Jefferson Y. Han. Low-cost multi-touch sensing through frustrated total internal reflection. In UIST '05: Proceedings of the 18th annual ACM symposium on User interface software and technology, pages 115--118, New York, NY, USA, 2005. ACM.
• 4: David Holman and Roel Vertegaal. Organic user interfaces: designing computers in any way, shape, or form. Commun. ACM, 51(6):48--55, 2008.
• 5: Hiroshi Ishii and Brygg Ullmer. Tangible bits: towards seamless interfaces between people, bits and atoms. In CHI '97: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 234--241, New York, NY, USA, 1997. ACM.
• 6: Takayuki Iwamoto, Mari Tatezono, Takayuki Hoshi, and Hiroyuki Shinoda. Airborne ultrasound tactile display. In SIGGRAPH '08: ACM SIGGRAPH 2008 new tech demos, pages 1--1, New York, NY, USA, 2008. ACM.
• 7: Kensei Jo, Yasuaki Kakehi, Kouta Minamizawa, Hideaki Nii, Naoki Kawakami, and Susumu Tachi. Arforce: a marker-based augmented reality system for force distribution input. In ACE '08: Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology, pages 160--165, New York, NY, USA, 2008. ACM.
• 8: Yasuaki Kakehi, Makoto Iida, and Takeshi Naemura. Tablescape plus: upstanding tiny displays on tabletop display. In SIGGRAPH '06: ACM SIGGRAPH 2006 Sketches, page 12, New York, NY, USA, 2006. ACM.
• 9: Kazuto Kamiyama, Hiroyuki Kajimoto, Kevin Vlack, Naoki Kawakami, Terukazu Mizota, and Susumu Tachi. Gelforce. In SIGGRAPH '04: ACM SIGGRAPH 2004 Emerging Technologies, page 5, New York, NY, USA, 2004. ACM.
• 10: Hideki Koike, Shinichiro Nagashima, Yasuto Nakanishi, and Yoichi Sato. Enhancedtable: Supporting a small meeting in ubiquitous and augmented environment. In Pacific Rim Conference on Multimedia, 2004.
• 11: Hideki Koike, Wataru Nishikawa, and Kentaro Fukuchi. Transparent 2-D markers on an LCD tabletop system. In CHI '09: Proceedings of the 27th International Conference on Human Factors in Computing Systems, pages 163--172, New York, NY, USA, 2009. ACM.
• 12: Hongen Liao, Yosuke Ito, Kiyoshi Matsumiya, Ken Masamune, and Takeyoshi Dohi. Object based image rendering and synthesis for computer generated integral videography. In SIGGRAPH '06: ACM SIGGRAPH 2006 Research posters, page 123, New York, NY, USA, 2006. ACM.
• 13: Nobuyuki Matsushita and Jun Rekimoto. Holowall: designing a finger, hand, body, and object sensitive wall. In UIST '97: Proceedings of the 10th annual ACM symposium on User interface software and technology, pages 209--210, New York, NY, USA, 1997. ACM.
• 14: Ben Piper, Carlo Ratti, and Hiroshi Ishii. Illuminating clay: a 3-D tangible interface for landscape analysis. In CHI '02: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 355--362, New York, NY, USA, 2002. ACM.
• 15: Jun Rekimoto. Smartskin: an infrastructure for freehand manipulation on interactive surfaces. In CHI '02: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 113--120, New York, NY, USA, 2002. ACM.
• 16: Katsunari Sato, Kouta Minamizawa, Naoki Kawakami, and Susumu Tachi. Haptic telexistence. In SIGGRAPH '07: ACM SIGGRAPH 2007 emerging technologies, page 10, New York, NY, USA, 2007. ACM.
• 17: Yoichi Sato, Yoshinori Kobayashi, and Hideki Koike. Fast tracking of hands and fingertips in infrared images for augmented desk interface. In FG '00: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000, page 462, Washington, DC, USA, 2000. IEEE Computer Society.
• 18: Brygg Ullmer and Hiroshi Ishii. The metadesk: models and prototypes for tangible user interfaces. In UIST '97: Proceedings of the 10th annual ACM symposium on User interface software and technology, pages 223--232, New York, NY, USA, 1997. ACM.
• 19: Malte Weiss, Julie Wagner, Yvonne Jansen, Roger Jennings, Ramsin Khoshabeh, James D. Hollan, and Jan Borchers. Slap widgets: bridging the gap between virtual and physical controls on tabletops. In CHI '09: Proceedings of the 27th international conference on Human factors in computing systems, pages 481--490, New York, NY, USA, 2009. ACM.
• 20: Ruigang Yang, Xinyu Huang, Sifang Li, and Christopher Jaynes. Toward the light field display: Autostereoscopic rendering via a cluster of projectors. IEEE Transactions on Visualization and Computer Graphics, 14(1):84--96, 2008.
• 21: Ben Piper Carlo Ratti Yao Wang, Assaf Biderman and Professor Hiroshi Ishii. SandScape. http://tangible.media.mit.edu/projects/sandscape/.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

General Terms:
Design, Human Factors

Keywords:
direct manipulation, interactive surfaces, multi-touch, photoelasticity, tabletop interfaces