A survey of design issues in spatial input

ABSTRACT

We present a survey of design issues for developing effective free-space three-dimensional (3D) user interfaces. Our survey is based upon previous work in 3D interaction, our experience in developing free-space interfaces, and our informal observations of test users. We illustrate our design issues using examples drawn from instances of 3D interfaces.For example, our first issue suggests that users have difficulty understanding three-dimensional space. We offer a set of strategies which may help users to better perceive a 3D virtual environment, including the use of spatial references, relative gesture, two-handed interaction, multisensory feedback, physical constraints, and head tracking. We describe interfaces which employ these strategies.Our major contribution is the synthesis of many scattered results, observations, and examples into a common framework. This framework should serve as a guide to researchers or systems builders who may not be familiar with design issues in spatial input. Where appropriate, we also try to identify areas in free-space 3D interaction which we see as likely candidates for additional research.An extended and annotated version of the references list for this paper is available on-line through mosaic at address http://uvacs.cs.virginia.edu/~kph2q/.

REFERENCES

• 1: Bernard D. Adelstein , Eric R. Johnston , Stephen R. Ellis, A testbed for characterizing dynamic response of virtual environment spatial sensors, Proceedings of the 5th annual ACM symposium on User interface software and technology, p.15-22, November 15-18, 1992, Monteray, California, United States [doi>10.1145/142621.142624]
• 2: Norman I. Badler , Kamran H. Manoochehri , David Baraff, Multi-dimensional input techniques and articulated figure positioning by multiple constraints, Proceedings of the 1986 workshop on Interactive 3D graphics, p.151-169, January 1987, Chapel Hill, North Carolina, United States [doi>10.1145/319120.319132]
• 3: Thomas Baudel , Michel Beaudouin-Lafon, Charade: remote control of objects using free-hand gestures, Communications of the ACM, v.36 n.7, p.28-35, July 1993 [doi>10.1145/159544.159562]
• 4: Larry Bergman , Henry Fuchs , Eric Grant , Susan Spach, Image rendering by adaptive refinement, ACM SIGGRAPH Computer Graphics, v.20 n.4, p.29-37, Aug. 1986
• 5: Eric A. Bier , Maureen C. Stone, Snap-dragging, ACM SIGGRAPH Computer Graphics, v.20 n.4, p.233-240, Aug. 1986
• 6: Eric A. Bier, Snap-dragging in three dimensions, ACM SIGGRAPH Computer Graphics, v.24 n.2, p.193-204, Mar. 1990
• 7: Eric A. Bier , Maureen C. Stone , Ken Pier , William Buxton , Tony D. DeRose, Toolglass and magic lenses: the see-through interface, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.73-80, September 1993 [doi>10.1145/166117.166126]
• 8: Richard A. Bolt, “Put-that-there”: Voice and gesture at the graphics interface, Proceedings of the 7th annual conference on Computer graphics and interactive techniques, p.262-270, July 14-18, 1980, Seattle, Washington, United States
• 9: Richard A. Bolt , Edward Herranz, Two-handed gesture in multi-modal natural dialog, Proceedings of the 5th annual ACM symposium on User interface software and technology, p.7-14, November 15-18, 1992, Monteray, California, United States [doi>10.1145/142621.142623]
• 10: Frederick P. Brooks, Jr., Walkthrough—a dynamic graphics system for simulating virtual buildings, Proceedings of the 1986 workshop on Interactive 3D graphics, p.9-21, January 1987, Chapel Hill, North Carolina, United States [doi>10.1145/319120.319122]
• 11: F. P. Brooks, Grasping reality through illusion—interactive graphics serving science, Proceedings of the SIGCHI conference on Human factors in computing systems, p.1-11, May 15-19, 1988, Washington, D.C., United States [doi>10.1145/57167.57168]
• 12: Frederick P. Brooks, Jr. , Ming Ouh-Young , James J. Batter , P. Jerome Kilpatrick, Project GROPEHaptic displays for scientific visualization, ACM SIGGRAPH Computer Graphics, v.24 n.4, p.177-185, Aug. 1990
• 13: Steve Bryson , Creon Levit, The Virtual Wind Tunnel, IEEE Computer Graphics and Applications, v.12 n.4, p.25-34, July 1992 [doi>10.1109/38.144824]
• 14: Jeff Butterworth , Andrew Davidson , Stephen Hench , Marc. T. Olano, 3DM: a three dimensional modeler using a head-mounted display, Proceedings of the 1992 symposium on Interactive 3D graphics, p.135-138, June 1992, Cambridge, Massachusetts, United States [doi>10.1145/147156.147182]
• 15: W. Buxton , B. Myers, A study in two-handed input, Proceedings of the SIGCHI conference on Human factors in computing systems, p.321-326, April 13-17, 1986, Boston, Massachusetts, United States
• 16: Stuart K. Card , Jock D. Mackinlay , George G. Robertson, The design space of input devices, Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, p.117-124, April 01-05, 1990, Seattle, Washington, United States [doi>10.1145/97243.97263]
• 17: Stuart K. Card , George G. Robertson , Jock D. Mackinlay, The information visualizer, an information workspace, Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, p.181-186, April 27-May 02, 1991, New Orleans, Louisiana, United States [doi>10.1145/108844.108874]
• 18: James C. Chung, A comparison of head-tracked and non-head-tracked steering modes in the targeting of radiotherapy treatment beams, Proceedings of the 1992 symposium on Interactive 3D graphics, p.193-196, June 1992, Cambridge, Massachusetts, United States [doi>10.1145/147156.147204]
• 19: P. R. Cohen , M. Dalrymple , D. B. Moran , F. C. Pereira , J. W. Sullivan, Synergistic use of direct manipulation and natural language, Proceedings of the SIGCHI conference on Human factors in computing systems: Wings for the mind, p.227-233, March 1989
• 20: Brookshire D. Conner , Scott S. Snibbe , Kenneth P. Herndon , Daniel C. Robbins , Robert C. Zeleznik , Andries van Dam, Three-dimensional widgets, Proceedings of the 1992 symposium on Interactive 3D graphics, p.183-188, June 1992, Cambridge, Massachusetts, United States [doi>10.1145/147156.147199]
• 21: Carolina Cruz-Neira , Daniel J. Sandin , Thomas A. DeFanti, Surround-screen projection-based virtual reality: the design and implementation of the CAVE, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.135-142, September 1993 [doi>10.1145/166117.166134]
• 22: Michael Deering, High resolution virtual reality, ACM SIGGRAPH Computer Graphics, v.26 n.2, p.195-202, July 1992
• 23: Steven Feiner , Blair MacIntyre , Marcus Haupt , Eliot Solomon, Windows on the world: 2D windows for 3D augmented reality, Proceedings of the 6th annual ACM symposium on User interface software and technology, p.145-155, December 1993, Atlanta, Georgia, United States [doi>10.1145/168642.168657]
• 24: Steven Feiner , Blair Macintyre , Dorée Seligmann, Knowledge-based augmented reality, Communications of the ACM, v.36 n.7, p.53-62, July 1993 [doi>10.1145/159544.159587]
• 25: Steven Feiner , Ari Shamash, Hybrid user interfaces: breeding virtually bigger interfaces for physically smaller computers, Proceedings of the 4th annual ACM symposium on User interface software and technology, p.9-17, November 11-13, 1991, Hilton Head, South Carolina, United States [doi>10.1145/120782.120783]
• 26: S.S. Fisher, M. McGreevy, J. Humphries, W. Robinett, "Virtual Interface Environment for Telepresence Applications," Oct. 1988, Proc. Human Factors Society 32nd Annual Meeting.
• 27: George W. Fitzmaurice, Situated information spaces and spatially aware palmtop computers, Communications of the ACM, v.36 n.7, p.39-49, July 1993 [doi>10.1145/159544.159566]
• 28: James D. Foley , Victor L. Wallace , Peggy Chan, The human factors of computer graphics interaction techniques, IEEE Computer Graphics and Applications, v.4 n.11, p.13-48, Nov. 1984
• 29: Tinsley A. Galyean , John F. Hughes, Sculpting: an interactive volumetric modeling technique, ACM SIGGRAPH Computer Graphics, v.25 n.4, p.267-274, July 1991
• 30: Gibson, J., The Ecological Approach to Visual Perception. Lawrence Efibaum, Hillsdale, NJ.
• 31: Gleicher, M., "Supporting Numerical Computations in Interactive Contexts," Graphics Interface '93.
• 32: Guiard, Y., "Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model," The Journal of Motor Behavior, 19 (4), 1987, pp. 486-517.
• 33: A. G. Hauptmann, Speech and gestures for graphic image manipulation, Proceedings of the SIGCHI conference on Human factors in computing systems: Wings for the mind, p.241-245, March 1989
• 34: Kenneth P. Herndon , Robert C. Zeleznik , Daniel C. Robbins , D. Brookshire Conner , Scott S. Snibbe , Andries van Dam, Interactive shadows, Proceedings of the 5th annual ACM symposium on User interface software and technology, p.1-6, November 15-18, 1992, Monteray, California, United States [doi>10.1145/142621.142622]
• 35: Ken Hinckley , Randy Pausch , John C. Goble , Neal F. Kassell, Passive real-world interface props for neurosurgical visualization, Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, p.452-458, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/191666.191821]
• 36: Hiroo Iwata, Artificial reality with force-feedback: development of desktop virtual space with compact master manipulator, ACM SIGGRAPH Computer Graphics, v.24 n.4, p.165-170, Aug. 1990
• 37: Robert J. K. Jacob , Linda E. Sibert, The perceptual structure of multidimensional input device selection, Proceedings of the SIGCHI conference on Human factors in computing systems, p.211-218, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142792]
• 38: Paul Kabbash , I. Scott MacKenzie , William Buxton, Human performance using computer input devices in the preferred and non-preferred hands, Proceedings of the SIGCHI conference on Human factors in computing systems, p.474-481, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169414]
• 39: Paul Kabbash , William Buxton , Abigail Sellen, Two-handed input in a compound task, Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, p.417-423, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/191666.191808]
• 40: Arie Kaufman , Roni Yagel, Tools for interaction in three dimensions, Proceedings of the third international conference on human-computer interaction, Vol.1 on Work with computers: organizational, management, stress and health aspects, p.468-475, September 1989, Boston, Massachusetts, United States
• 41: Myron W. Krueger, Environmental technology: making the real world virtual, Communications of the ACM, v.36 n.7, p.36-37, July 1993 [doi>10.1145/159544.159563]
• 42: Jiandong Liang , Chris Shaw , Mark Green, On temporal-spatial realism in the virtual reality environment, Proceedings of the 4th annual ACM symposium on User interface software and technology, p.19-25, November 11-13, 1991, Hilton Head, South Carolina, United States [doi>10.1145/120782.120784]
• 43: Liang, J., Green, M., "JDCAD: A Highly Interactive 3D Modeling System," 3rd International Conference on CAD and Computer Graphics, Beijing, China, Aug. 1993, 217-222.
• 44: Jock D. Mackinlay , Stuart K. Card , George G. Robertson, Rapid controlled movement through a virtual 3D workspace, ACM SIGGRAPH Computer Graphics, v.24 n.4, p.171-176, Aug. 1990
• 45: Michael McKenna, Interactive viewpoint control and three-dimensional operations, Proceedings of the 1992 symposium on Interactive 3D graphics, p.53-56, June 1992, Cambridge, Massachusetts, United States [doi>10.1145/147156.147163]
• 46: Margaret Minsky , Ouh-young Ming , Oliver Steele , Frederick P. Brooks, Jr. , Max Behensky, Feeling and seeing: issues in force display, ACM SIGGRAPH Computer Graphics, v.24 n.2, p.235-241, Mar. 1990
• 47: Jakob Nielsen, Noncommand user interfaces, Communications of the ACM, v.36 n.4, p.83-99, April 1993 [doi>10.1145/255950.153582]
• 48: Norman D., The Design of Everyday Things. Doubleday: New York, New York, 1990.
• 49: Eben Ostby, Describing free-form 3D surfaces for animation, Proceedings of the 1986 workshop on Interactive 3D graphics, p.251-258, January 1987, Chapel Hill, North Carolina, United States [doi>10.1145/319120.319138]
• 50: Pausch, R., "Support for Rapid Prototyping of Twoand Three-Dimensional User Interfaces," Proposal for ARPA BAA 93-42. Comp. Science Department, University of Virginia, March, 1994.
• 51: Pausch, R., Shackelford, M. A., Proffitt, D., "A User Study Comparing Head-Mounted and Stationary Displays," Proc. IEEE Symposium on Research Frontiers in Virtual Reality, Oct. 1993.
• 52: Pixsys Inc., 3522 22nd St., Boulder, CO 80304. [phone].
• 53: Polhemus Navigation Sciences, Inc., P. O. Box 560, Colchester, VT 05446. [phone].
• 54: Emanuel Sachs , Andrew Roberts , David Stoops, 3-Draw: A Tool for Designing 3D Shapes, IEEE Computer Graphics and Applications, v.11 n.6, p.18-26, November 1991 [doi>10.1109/38.103389]
• 55: Christopher Schmandt, Spatial input/display correspondence in a stereoscopic computer graphic work station, ACM SIGGRAPH Computer Graphics, v.17 n.3, p.253-261, July 1983
• 56: Andrew Sears , Catherine Plaisant , Ben Shneiderman, A new era for high precision touchscreens, Advances in human-computer interaction (vol. 3), Ablex Publishing Corp., Norwood, NJ, 1993
• 57: Shepard, R. N., Metzler, J., "Mental Rotation of Three- Dimensional Objects," Science, Vol. 171, 1971, pp. 701-703.
• 58: Spaceball Technologies, Inc. [phone].
• 59: Stoakley, R., Pausch, R., "Virtual Kit of Parts," unpublished manuscript, available through mosaic at http ://uvacs. cs. vi rg inia. edu/~ rws2v/p linth, html.
• 60: D. J. Sturman , D. Zeltzer , S. Pieper, Hands-on interaction with virtual environments, Proceedings of the 2nd annual ACM SIGGRAPH symposium on User interface software and technology, p.19-24, November 13-15, 1989, Williamsburg, Virginia, United States [doi>10.1145/73660.73663]
• 61: I.E. Sutherland, "A Head-mounted Three Dimensional Display," Proc. the Fall Joint Computer Conference, 1968, pp. 757-764.
• 62: H. Takemura , A. Tomono , Y. Kobayashi, An evaluation of 3-D object pointing using a field sequential stereoscopic display, Proceedings on Graphics interface '88, p.112-118, December 1989, Edmonton, Alberta, Canada
• 63: Russell M. Taylor , Warren Robinett , Vernon L. Chi , Frederick P. Brooks, Jr. , William V. Wright , R. Stanley Williams , Erik J. Snyder, The nanomanipulator: a virtual-reality interface for a scanning tunneling microscope, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.127-134, September 1993 [doi>10.1145/166117.166133]
• 64: C. Ware, Using hand position for virtual object placement, The Visual Computer: International Journal of Computer Graphics, v.6 n.5, p.245-253, 1990 [doi>10.1007/BF01900747]
• 65: Colin Ware , Steven Osborne, Exploration and virtual camera control in virtual three dimensional environments, ACM SIGGRAPH Computer Graphics, v.24 n.2, p.175-183, Mar. 1990
• 66: Colin Ware , Kevin Arthur , Kellogg S. Booth, Fish tank virtual reality, Proceedings of the SIGCHI conference on Human factors in computing systems, p.37-42, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169066]
• 67: Pierre Wellner, Interacting with paper on the DigitalDesk, Communications of the ACM, v.36 n.7, p.87-96, July 1993 [doi>10.1145/159544.159630]
• 68: Shumin Zhai , William Buxton , Paul Milgram, The “Silk Cursor”: investigating transparency for 3D target acquisition, Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, p.459-464, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/191666.191822]
• 69: Thomas G. Zimmerman , Jaron Lanier , Chuck Blanchard , Steve Bryson , Young Harvill, A hand gesture interface device, Proceedings of the SIGCHI/GI conference on Human factors in computing systems and graphics interface, p.189-192, April 05-09, 1987, Toronto, Ontario, Canada

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Virtual reality

General Terms:
Design, Human Factors

Keywords:
3D interaction, ergonomics of virtual manipulation, haptic input, spatial input, two-handed input, virtual reality