Providing dynamically changeable physical buttons on a visual display

ABSTRACT

Physical buttons have the unique ability to provide low-attention and vision-free interactions through their intuitive tactile clues. Unfortunately, the physicality of these interfaces makes them static, limiting the number and types of user interfaces they can support. On the other hand, touch screen technologies provide the ultimate interface flexibility, but offer no inherent tactile qualities. In this paper, we describe a technique that seeks to occupy the space between these two extremes - offering some of the flexibility of touch screens, while retaining the beneficial tactile properties of physical interfaces.

The outcome of our investigations is a visual display that contains deformable areas, able to produce physical buttons and other interface elements. These tactile features can be dynamically brought into and out of the interface, and otherwise manipulated under program control. The surfaces we describe provide the full dynamics of a visual display (through rear projection) as well as allowing for multitouch input (though an infrared lighting and camera setup behind the display). To illustrate the tactile capabilities of the surfaces, we describe a number of variations we uncovered in our exploration and prototyping. These go beyond simple on/off actuation and can be combined to provide a range of different possible tactile expressions. A preliminary user study indicates that our dynamic buttons perform much like physical buttons in tactile search tasks.

REFERENCES

• 1: Kenneth Majlund Ba h , Mads Gregers Jæger , Mikael B. Skov , Nils Gram Thomassen, You can touch, but you can't look: interacting with in-vehicle systems, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357233]
• 2: Benali-Khoudja, M., Hafez, M., Alexandre, J. M., Kheddar, A. Tactile Interfaces: A State of the Art Survey. In Proc. ISR '04. 721--726.
• 3: Hrvoje Benko , Andrew D. Wilson , Ravin Balakrishnan, Sphere: multi-touch interactions on a spherical display, Proceedings of the 21st annual ACM symposium on User interface software and technology, October 19-22, 2008, Monterey, CA, USA [doi>10.1145/1449715.1449729]
• 4: Stephen Brewster , Lorna M. Brown, Tactons: structured tactile messages for non-visual information display, Proceedings of the fifth conference on Australasian user interface, p.15-23, January 01, 2004, Dunedin, New Zealand
• 5: Marcelo Coelho , Pattie Maes, Sprout I/O: a texturally rich interface, Proceedings of the 2nd international conference on Tangible and embedded interaction, February 18-20, 2008, Bonn, Germany [doi>10.1145/1347390.1347440]
• 6: Marcelo Coelho , Hiroshi Ishii , Pattie Maes, Surflex: a programmable surface for the design of tangible interfaces, CHI '08 extended abstracts on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1358628.1358869]
• 7: Paul Dietz , Darren Leigh, DiamondTouch: a multi-user touch technology, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502389]
• 8: Mario Enriquez , Oleg Afonin , Brent Yager , Karon Maclean, A pneumatic tactile alerting system for the driving environment, Proceedings of the 2001 workshop on Perceptive user interfaces, November 15-16, 2001, Orlando, Florida [doi>10.1145/971478.971506]
• 9: Masaaki Fukumoto , Toshiaki Sugimura, Active click: tactile feedback for touch panels, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634141]
• 10: Jefferson Y. Han, Low-cost multi-touch sensing through frustrated total internal reflection, Proceedings of the 18th annual ACM symposium on User interface software and technology, October 23-26, 2005, Seattle, WA, USA [doi>10.1145/1095034.1095054]
• 11: Yuki Hashimoto , Hiroyuki Kajimoto, A novel interface to present emotional tactile sensation to a palm using air pressure, CHI '08 extended abstracts on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1358628.1358748]
• 12: Hiroo Iwata , Hiroaki Yano , Fumitaka Nakaizumi , Ryo Kawamura, Project FEELEX: adding haptic surface to graphics, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.469-476, August 2001 [doi>10.1145/383259.383314]
• 13: Seoktae Kim , Hyunjung Kim , Boram Lee , Tek-Jin Nam , Woohun Lee, Inflatable mouse: volume-adjustable mouse with air-pressure-sensitive input and haptic feedback, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357090]
• 14: Kodama, S. and M. Takeno. Protrude, Flow. (Electronic Arts and Animation Catalogue) In Proc. SIGGRAPH '01. ACM Press (2001), 138.
• 15: Laitinen, P. Tactile Touch Screen. International Application PCT/EP2006/009377, filed 27 September 2006.
• 16: Johnny C. Lee , Paul H. Dietz , Darren Leigh , William S. Yerazunis , Scott E. Hudson, Haptic pen: a tactile feedback stylus for touch screens, Proceedings of the 17th annual ACM symposium on User interface software and technology, October 24-27, 2004, Santa Fe, NM, USA [doi>10.1145/1029632.1029682]
• 17: Kevin A. Li , Patrick Baudisch , William G. Griswold , James D. Hollan, Tapping and rubbing: exploring new dimensions of tactile feedback with voice coil motors, Proceedings of the 21st annual ACM symposium on User interface software and technology, October 19-22, 2008, Monterey, CA, USA [doi>10.1145/1449715.1449744]
• 18: Nobuyuki Matsushita , Jun Rekimoto, HoloWall: designing a finger, hand, body, and object sensitive wall, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.209-210, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263549]
• 19: Microsoft surface. http://www.microsoft.com/surface/
• 20: Masashi Nakatani , Hiroyuki Kajimoto , Naoki Kawakami , Susumu Tachi, Tactile sensation with high-density pin-matrix, Proceedings of the 2nd symposium on Applied perception in graphics and visualization, August 26-28, 2005, A Coroña, Spain [doi>10.1145/1080402.1080452]
• 21: Andrew Nashel , Sharif Razzaque, Tactile virtual buttons for mobile devices, CHI '03 extended abstracts on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/765891.766032]
• 22: Otsuka, K. and Wayman, C. M. Eds. Shape Memory Materials. Cambridge University Press, New York, 1999.
• 23: Overholt, D., Pasztor, E., Mazalek, A. A Multipurpose Array of Tactile Rods for Interactive sXpression (Abstracts and Applications), In Proc. SIGGRAPH '01. ACM Press (2001).
• 24: Ivan Poupyrev , Shigeaki Maruyama , Jun Rekimoto, Ambient touch: designing tactile interfaces for handheld devices, Proceedings of the 15th annual ACM symposium on User interface software and technology, October 27-30, 2002, Paris, France [doi>10.1145/571985.571993]
• 25: Ivan Poupyrev , Tatsushi Nashida , Shigeaki Maruyama , Jun Rekimoto , Yasufumi Yamaji, Lumen: interactive visual and shape display for calm computing, ACM SIGGRAPH 2004 Emerging technologies, August 08-12, 2004, Los Angeles, California [doi>10.1145/1186155.1186173]
• 26: Ivan Poupyrev , Tatsushi Nashida , Makoto Okabe, Actuation and tangible user interfaces: the Vaucanson duck, robots, and shape displays, Proceedings of the 1st international conference on Tangible and embedded interaction, February 15-17, 2007, Baton Rouge, Louisiana [doi>10.1145/1226969.1227012]
• 27: Gonzalo Ramos , Matthew Boulos , Ravin Balakrishnan, Pressure widgets, Proceedings of the SIGCHI conference on Human factors in computing systems, p.487-494, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985754]
• 28: Jun Rekimoto , Carsten Schwesig, PreSenseII: bi-directional touch and pressure sensing interactions with tactile feedback, CHI '06 extended abstracts on Human factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1125451.1125685]
• 29: Rockwell, T.H. Spare Visual Capacity in Driving Revisited: New Empirical Results of an Old Idea, in Vision min Vehicles II. Elsevier (1988), 317-324.
• 30: Wagner, C. R, Lederman, S. J., Howe, R.D. A Tactile Shape Display Using RC Servomotors. Electronic Journal of Haptics Research, 3, 4.
• 31: Wickens, C.D. and Hollands, J.G. Engineering Psychology and Human Performance. Prentice Hall, 2000.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.m Miscellaneous

General Terms:
Human Factors

Keywords:
barometric, dashboard, dynamic buttons, eyes-free, haptic, input, multitouch, physical interfaces, pneumatic, pressure, programmatically controlled, rear projection, shape displays, tactile