Mechanical constraints as computational constraints in tabletop tangible interfaces

ABSTRACT

This paper presents a new type of human-computer interface called Pico (Physical Intervention in Computational Optimization) based on mechanical constraints that combines some of the tactile feedback and affordances of mechanical systems with the abstract computational power of modern computers. The interface is based on a tabletop interaction surface that can sense and move small objects on top of it. The positions of these physical objects represent and control parameters inside a software application, such as a system for optimizing the configuration of radio towers in a cellular telephone network. The computer autonomously attempts to optimize the network, moving the objects on the table as it changes their corresponding parameters in software. As these objects move, the user can constrain their motion with his or her hands, or many other kinds of physical objects. The interface provides ample opportunities for improvisation by allowing the user to employ a rich variety of everyday physical objects as mechanical constraints. This approach leverages the user's mechanical intuition for how objects respond to physical forces. As well, it allows the user to balance the numerical optimization performed by the computer with other goals that are difficult to quantify. Subjects in an evaluation were more effective at solving a complex spatial layout problem using this system than with either of two alternative interfaces that did not feature actuation.

REFERENCES

• 1: David Anderson , Emily Anderson , Neal Lesh , Joe Marks , Brian Mirtich , David Ratajczak , Kathy Ryall, Human-Guided Simple Search, Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence, p.209-216, July 30-August 03, 2000
• 2: Frederick P. Brooks, Jr. , Ming Ouh-Young , James J. Batter , P. Jerome Kilpatrick, Project GROPEHaptic displays for scientific visualization, Proceedings of the 17th annual conference on Computer graphics and interactive techniques, p.177-185, September 1990, Dallas, TX, USA
• 3: Fang Chen , Peter Eades , Julien Epps , Serge Lichman , Benjamin Close , Peter Hutterer , Masa Takatsuka , Bruce Thomas , Mike Wu, ViCAT: Visualisation and Interaction on a Collaborative Access Table, Proceedings of the First IEEE International Workshop on Horizontal Interactive Human-Computer Systems, p.59-62, January 05-07, 2006 [doi>10.1109/TABLETOP.2006.36]
• 4: Katherine Everitt , Chia Shen , Kathy Ryall , Clifton Forlines, MultiSpace: Enabling Electronic Document Micro-mobility in Table-Centric, Multi-Device Environments, Proceedings of the First IEEE International Workshop on Horizontal Interactive Human-Computer Systems, p.27-34, January 05-07, 2006 [doi>10.1109/TABLETOP.2006.23]
• 5: George W. Fitzmaurice , William Buxton, Graspable user interfaces, University of Toronto, Toronto, Ont., Canada, 1996
• 6: Morten Fjeld , Fred Voorhorst , Martin Bichsel , Helmut Krueger , Matthias Rauterberg, Navigation methods for an augmented reality system, CHI '00 extended abstracts on Human factors in computing systems, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/633292.633298]
• 7: Morten Fjeld , Kristina Lauche , Martin Bichsel , Fred Voorhorst , Helmut Krueger , Matthias Rauterberg, Physical and Virtual Tools: ActivityTheory Applied to the Design of Groupware, Computer Supported Cooperative Work, v.11 n.1-2, p.153-180, 2002 [doi>10.1023/A:1015269228596]
• 8: Morten Fjeld , Sissel Guttormsen Schär , Domenico Signorello , Helmut Krueger, Alternative Tools for Tangible Interaction: A Usability Evaluation, Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR'02), p.157, September 30-October 01, 2002
• 9: Goldstein, H., Poole. C., Safko., J., Classical Mechanics. Addison Wesley, 2002. ISBN: 0201657023
• 10: Hiroshi Ishii , Brygg Ullmer, Tangible bits: towards seamless interfaces between people, bits and atoms, Proceedings of the SIGCHI conference on Human factors in computing systems, p.234-241, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258715]
• 11: Yasuaki Kakehi , Takero Hosomi , Makoto Iida , Takeshi Naemura , Mitsunori Matsushita, Transparent Tabletop Interface for Multiple Users on Lumisight Table, Proceedings of the First IEEE International Workshop on Horizontal Interactive Human-Computer Systems, p.143-150, January 05-07, 2006 [doi>10.1109/TABLETOP.2006.34]
• 12: Kirsh, D. and Maglio, P. On Distinguishing Epistemic from Pragmatic Actions. Cognitive Science. (1995).
• 13: Minoru KOJIMA , Maki SUGIMOTO , Akihiro NAKAMURA , Masahiro TOMITA , Masahiko INAMI , Hideaki NII, Augmented Coliseum: An Augmented Game Environment with Small Vehicles, Proceedings of the First IEEE International Workshop on Horizontal Interactive Human-Computer Systems, p.3-8, January 05-07, 2006 [doi>10.1109/TABLETOP.2006.3]
• 14: J. Marks , B. Andalman , P. A. Beardsley , W. Freeman , S. Gibson , J. Hodgins , T. Kang , B. Mirtich , H. Pfister , W. Ruml , K. Ryall , J. Seims , S. Shieber, Design galleries: a general approach to setting parameters for computer graphics and animation, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.389-400, August 1997 [doi>10.1145/258734.258887]
• 15: Metropolis,N., Rosenbluth, A., Rosenbluth, M. Teller, E. "Equation of State Calculations by Fast Computing Machines", J. Chem. Phys.,21, 6, 1087--1092, 1953.
• 16: Motorola, Netplan RF engineering user's manual version 6.1, Tech. Rep., Motorola, May 2002.
• 17: Gian Pangaro , Dan Maynes-Aminzade , Hiroshi Ishii, The actuated workbench: computer-controlled actuation in tabletop tangible interfaces, Proceedings of the 15th annual ACM symposium on User interface software and technology, October 27-30, 2002, Paris, France [doi>10.1145/571985.572011]
• 18: James Patten , Hiroshi Ishii , Jim Hines , Gian Pangaro, Sensetable: a wireless object tracking platform for tangible user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, p.253-260, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365112]
• 19: James Patten , Ben Recht , Hiroshi Ishii, Audiopad: a tag-based interface for musical performance, Proceedings of the 2002 conference on New interfaces for musical expression, May 24-26, 2002, Dublin, Ireland
• 20: Ben Piper , Carlo Ratti , Hiroshi Ishii, Illuminating clay: a 3-D tangible interface for landscape analysis, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503439]
• 21: Reznik, D., Canny, J., C'mon Part, do the Local Motion!, 2001 IEEE International Conference on Robotics & Automation (ICRA), Seoul, Korea, IEEE Publisher (2001).
• 22: Richard, C., and Cutkosky, M., The Effects of Real and Computer Generated Friction on Human Performance in a Targeting Task, Proceedings of the ASME Dynamic Systems and Control Division, vol. 69--2, (2000), pp. 1101--1108.
• 23: Rosen, J., Hannaford, B., MacFarlane, M., and Sinanan, M. Force Controlled and Teleoperated Endoscopic Grasper for Minimally Invasive Surgery -- Experimental Performance Evaluation, IEEE Transactions on Biomedical Engineering, vol. 46, no. 10, IEEE Publisher (1999). pp. 1212--1221.
• 24: Dan Rosenfeld , Michael Zawadzki , Jeremi Sudol , Ken Perlin, Physical Objects as Bidirectional User Interface Elements, IEEE Computer Graphics and Applications, v.24 n.1, p.44-49, January 2004 [doi>10.1109/MCG.2004.1255808]
• 25: Ryall, K., Marks, J., Shieber, S., An Interactive Constraint--Based System for Drawing Graphs. Mitsubishi Electric Research Laboratory technical report TR-97-15.
• 26: Stacey D. Scott , Neal Lesh , Gunnar W. Klau, Investigating human-computer optimization, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503405]
• 27: Ehud Sharlin , Benjamin Watson , Yoshifumi Kitamura , Fumio Kishino , Yuichi Itoh, On tangible user interfaces, humans and spatiality, Personal and Ubiquitous Computing, v.8 n.5, p.338-346, September 2004 [doi>10.1007/s00779-004-0296-5]
• 28: Stuart M. Shieber, A call for collaborative interfaces, ACM Computing Surveys (CSUR), v.28 n.4es, Dec. 1996 [doi>10.1145/242224.242408]
• 29: Brygg Ullmer , Hiroshi Ishii , Robert J. K. Jacob, Token+constraint systems for tangible interaction with digital information, ACM Transactions on Computer-Human Interaction (TOCHI), v.12 n.1, p.81-118, March 2005 [doi>10.1145/1057237.1057242]
• 30: John Underkoffler , Hiroshi Ishii, Urp: a luminous-tangible workbench for urban planning and design, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.386-393, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303114]
• 31: Waters, C.D.J. Interactive Vehicle Routing. Journal of the Operational Research Society, 35(9), (1984), pp. 821--826.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation); Haptic I/O

General Terms:
Human Factors

Keywords:
actuation, improvisation, interactive surface, physical interaction, tangible interfaces