TouchLight: an imaging touch screen and display for gesture-based interaction

ABSTRACT

A novel touch screen technology is presented. TouchLight uses simple image processing techniques to combine the output of two video cameras placed behind a semi-transparent plane in front of the user. The resulting image shows objects that are on the plane. This technique is well suited for application with a commercially available projection screen material (DNP HoloScreen) which permits projection onto a transparent sheet of acrylic plastic in normal indoor lighting conditions. The resulting touch screen display system transforms an otherwise normal sheet of acrylic plastic into a high bandwidth input/output surface suitable for gesture-based interaction. Image processing techniques are detailed, and several novel capabilities of the system are outlined.

REFERENCES

• 1: Bill Buxton , Tom Moran, EuroPARC's integrated interactive intermedia facility (IIIF): early experiences, Proceedings of the IFIP WG 8.4 confernece on Multi-user interfaces and applications, p.11-34, January 1990, Heraklion, Crete, Greece
• 2: de la Hammette, P., P. Lukowicz, G. Tröster, T. Svoboda, Fingermouse: A Wearable Hand Tracking System. in Ubicomp 2003: Ubiquitous Computing, (2002).
• 3: Paul Dietz , Darren Leigh, DiamondTouch: a multi-user touch technology, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502389]
• 4: Ferscha A., M.K., DigiScope: An Invisible Worlds Window. in Adjunct Proceedings, The Fifth International Conference on Ubiquitous Computing, (Seattle, 2003), 261--264.
• 5: Berthold K. Horn, Robot Vision, McGraw-Hill Higher Education, 1986
• 6: Hiroshi Ishii , Minoru Kobayashi, ClearBoard: a seamless medium for shared drawing and conversation with eye contact, Proceedings of the SIGCHI conference on Human factors in computing systems, p.525-532, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142977]
• 7: Kang, S.B. Radial Distortion Snakes. IEICE Transactions on Information and Systems, E84-D (12). 1603--1611.
• 8: Scott R. Klemmer , Mark W. Newman , Ryan Farrell , Mark Bilezikjian , James A. Landay, The designers' outpost: a tangible interface for collaborative web site, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502350]
• 9: Nobuyuki Matsushita , Jun Rekimoto, HoloWall: designing a finger, hand, body, and object sensitive wall, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.209-210, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263549]
• 10: Joseph A. Paradiso , Che King Leo , Nisha Checka , Kaijen Hsiao, Passive acoustic knock tracking for interactive windows, CHI '02 extended abstracts on Human factors in computing systems, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/506443.506570]
• 11: Meredith Ringel , Kathy Ryall , Chia Shen , Clifton Forlines , Frederic Vernier, Release, relocate, reorient, resize: fluid techniques for document sharing on multi-user interactive tables, CHI '04 extended abstracts on Human factors in computing systems, April 24-29, 2004, Vienna, Austria [doi>10.1145/985921.986085]
• 12: Chia Shen , Frédéric D. Vernier , Clifton Forlines , Meredith Ringel, DiamondSpin: an extensible toolkit for around-the-table interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, p.167-174, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985714]
• 13: Brygg Ullmer , Hiroshi Ishii, The metaDESK: models and prototypes for tangible user interfaces, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.223-232, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263551]
• 14: Wren, C.R., Y. A. Ivanov, Volumetric Operations with Surface Margins. in Computer Vision and Pattern Recognition: Technical Sketches, (2001).
• 15: Mike Wu , Ravin Balakrishnan, Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.193-202, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964718]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.9 Applications

General Terms:
Algorithms, Design, Human Factors

Keywords:
computer human interaction, computer vision, displays, gesture recognition, videoconferencing