Two-handed virtual manipulation

ABSTRACT

We discuss a two-handed user interface designed to support three-dimesional neurosurgical visualization. By itself, this system is a “point design,” an example of an advanced user interface technique. In this work, we argue that in order to understand why interaction techniques do or do not work, and to suggest possibilities for new techniques, it is important to move beyond point design and to introduce careful scientific measurement of human behavioral principles. In particular, we argue that the common-sense viewpoint that “two hands save time by working in parallel” may not always be an effective way to think about two-handed interface design because the hands do not necessarily work in parallel (there is a structure to two-handed manipulation) and because two hands do more than just save time over one hand (two hands provide the user with more information and can structure how the user thinks about a task). To support these claims, we present an interface design developed in collaboration with neurosurgeons which has undergone extensive informal usability testing, as well as a pair of formal experimental studies which investigate behavioral aspects of two-handed virtual object manipulation. Our hope is that this discussion will help others to apply the lessons in our neurosurgery application to future two-handed user interface designs.

REFERENCES

• 1: ANNETT, J., ANNETT, M., HUDSON, P., AND TURNER, A. 1979. The control of movement in the preferred and non-preferred hands. Q. J. Exp. Psychol. 31,641-652.
• 2: Jeff Johnson , Teresa L. Roberts , William Verplank , David C. Smith , Charles H. Irby , Marian Beard , Kevin Mackey, The emergence of graphical user interfaces, Human-computer interaction: toward the year 2000, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1995
• 3: Eric A. Bier , Maureen C. Stone , Ken Pier , William Buxton , Tony D. DeRose, Toolglass and magic lenses: the see-through interface, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.73-80, September 1993 [doi>10.1145/166117.166126]
• 4: Richard A. Bolt , Edward Herranz, Two-handed gesture in multi-modal natural dialog, Proceedings of the 5th annual ACM symposium on User interface software and technology, p.7-14, November 15-18, 1992, Monteray, California, United States [doi>10.1145/142621.142623]
• 5: Edward G. Britton , James S. Lipscomb , Michael E. Pique, Making nested rotations convenient for the user, ACM SIGGRAPH Computer Graphics, v.12 n.3, p.222-227, August 1978
• 6: F. P. Brooks, Grasping reality through illusion—interactive graphics serving science, Proceedings of the SIGCHI conference on Human factors in computing systems, p.1-11, May 15-19, 1988, Washington, D.C., United States [doi>10.1145/57167.57168]
• 7: BUXTON, W. 1986. Chunking and phrasing and the design of human-computer dialogues. In Proceedings of the IFIP World Computer Congress on Information Processing 86 (Dublin, Ireland, Sept. 1-5), H.-J. Kugler, Ed. North-Holland Publishing Co., Amsterdam, The Netherlands, 475-480.
• 8: BUXTON, W. 1990. The pragmatics of haptic input. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI '90, Seattle, WA, Apr. 1-5). ACM, New York, NY.
• 9: W. Buxton , B. Myers, A study in two-handed input, Proceedings of the SIGCHI conference on Human factors in computing systems, p.321-326, April 13-17, 1986, Boston, Massachusetts, United States
• 10: Stuart K. Card , Thomas P. Moran, User technology: from pointing to pondering, Human-computer interaction: toward the year 2000, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1995
• 11: CARD, S., ENGLISH, W., AND BURR, B. 1978. Evaluation of mouse, rate-controlled isometric joystick, step keys, and text keys for text selection on a CRT. Ergonomics 21,601-613.
• 12: Stéphane Chatty, Extending a graphical toolkit for two-handed interaction, Proceedings of the 7th annual ACM symposium on User interface software and technology, p.195-204, November 02-04, 1994, Marina del Rey, California, United States [doi>10.1145/192426.192500]
• 13: Stephané Chatty, Issues and experience in designing two-handed interaction, Conference companion on Human factors in computing systems, p.253-254, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/259963.260470]
• 14: CREMER, M. AND ASHTON, R. 1981. Motor performance and concurrent cognitive tasks. J. Motor Behav. 13, 187-196.
• 15: Lawrence D. Cutler , Bernd Fröhlich , Pat Hanrahan, Two-handed direct manipulation on the responsive workbench, Proceedings of the 1997 symposium on Interactive 3D graphics, p.107-114, April 27-30, 1997, Providence, Rhode Island, United States [doi>10.1145/253284.253315]
• 16: Michael Deering, High resolution virtual reality, ACM SIGGRAPH Computer Graphics, v.26 n.2, p.195-202, July 1992
• 17: George W. Fitzmaurice , William Buxton, An empirical evaluation of graspable user interfaces: towards specialized, space-multiplexed input, Proceedings of the SIGCHI conference on Human factors in computing systems, p.43-50, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258578]
• 18: George W. Fitzmaurice , Hiroshi Ishii , William A. S. Buxton, Bricks: laying the foundations for graspable user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, p.442-449, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223964]
• 19: Tinsley A. Galyean , John F. Hughes, Sculpting: an interactive volumetric modeling technique, ACM SIGGRAPH Computer Graphics, v.25 n.4, p.267-274, July 1991
• 20: GIBSON, J. 1962. Observations on active touch. Psychol. Rev. 69, 6, 477-491.
• 21: John C. Goble , Ken Hinckley , Randy Pausch , John W. Snell , Neal F. Kassell, Two-Handed Spatial Interface Tools for Neurosurgical Planning, Computer, v.28 n.7, p.20-26, July 1995 [doi>10.1109/2.391037]
• 22: GOBLE, J., SNELL, J., HINCKLEY, K., AND KASSELL, N. 1994. A real-time system for 3D neurosurgical planning. In Proceedings of the Conference on Visualization in Biomedical Computing (Rochester, MN, Sept.). Proceedings of SHE--The International Society for Optical Engineering, vol. 2359. SPIE, Bellingham, WA, 552-573.
• 23: Maarten W. Gribnau , James M. Hennessey, Comparing single- and two-handed 3D input for a 3D object assembly task, CHI 98 conference summary on Human factors in computing systems, p.233-234, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/286498.286720]
• 24: GUIARD, Y. 1987. Asymmetric division of labor in human skilled bimanual action: The kinematic chain as a model. J. Motor Behav. 19, 4, 486-517.
• 25: GUIARD, Y. 1989. Failure to sing the left-hand part of the score during piano performance. J. Music Perception 6, 3, 299-314.
• 26: GUIARD, Y. AND FERRAND, T. 1995. Asymmetry in bimanual skills. In Manual Asymmetries in Motor Performance, Elliott, and Roy, Eds. CRC Press, Inc., Boca Raton, FL.
• 27: HALPERN, D. 1992. Sex Differences in Cognitive Ability. Lawrence Erlbaum Associates, Inc., Mahwah, NJ.
• 28: A. G. Hauptmann, Speech and gestures for graphic image manipulation, Proceedings of the SIGCHI conference on Human factors in computing systems: Wings for the mind, p.241-245, March 1989
• 29: Ken Hinckley, Haptic issues for virtual manipulation, University of Virginia, Charlottesville, VA, 1997
• 30: HINCKLEY, K., GOBLE, J., PAUSCH, R., AND KASSELL, N. 1995. New applications for the touchscreen in 2D and 3D medical imaging workstations. In Proceedings of the Conference on Medical Imaging 1995: Image Display. Proceedings of SPIE--The International Society for Optical Engineering, vol. 2431. SPIE, Bellingham, WA, 561-570.
• 31: Ken Hinckley , Randy Pausch , John C. Goble , Neal F. Kassell, Passive real-world interface props for neurosurgical visualization, Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, p.452-458, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/191666.191821]
• 32: Ken Hinckley , Randy Pausch , John C. Goble , Neal F. Kassell, A survey of design issues in spatial input, Proceedings of the 7th annual ACM symposium on User interface software and technology, p.213-222, November 02-04, 1994, Marina del Rey, California, United States [doi>10.1145/192426.192501]
• 33: Ken Hinckley , Randy Pausch , Dennis Proffitt, Attention and visual feedback: the bimanual frame of reference, Proceedings of the 1997 symposium on Interactive 3D graphics, p.121-ff., April 27-30, 1997, Providence, Rhode Island, United States [doi>10.1145/253284.253318]
• 34: Ken Hinckley , Randy Pausch , Dennis Proffitt , James Patten , Neal Kassell, Cooperative bimanual action, Proceedings of the SIGCHI conference on Human factors in computing systems, p.27-34, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258571]
• 35: Ken Hinckley , Joe Tullio , Randy Pausch , Dennis Proffitt , Neal Kassell, Usability analysis of 3D rotation techniques, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.1-10, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263408]
• 36: HONDA, H. 1982. Rightward superiority of eye movements in a bimanual aiming task. Q.J. Exp. Psychol. 34A, 499-513.
• 37: Hiroshi Ishii , Brygg Ullmer, Tangible bits: towards seamless interfaces between people, bits and atoms, Proceedings of the SIGCHI conference on Human factors in computing systems, p.234-241, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258715]
• 38: Robert J. K. Jacob , Linda E. Sibert , Daniel C. McFarlane , M. Preston Mullen, Jr., Integrality and separability of input devices, ACM Transactions on Computer-Human Interaction (TOCHI), v.1 n.1, p.3-26, March 1994 [doi>10.1145/174630.174631]
• 39: Paul Kabbash , William Buxton , Abigail Sellen, Two-handed input in a compound task, Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, p.417-423, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/191666.191808]
• 40: Paul Kabbash , I. Scott MacKenzie , William Buxton, Human performance using computer input devices in the preferred and non-preferred hands, Proceedings of the SIGCHI conference on Human factors in computing systems, p.474-481, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169414]
• 41: KIYOKAWA, K., TAKEMURA, H., KATAYAMA, Y., IWASA, H., AND YOKOYA, N. 1996. VLEGO: A simple two-handed modeling environment based on toy blocks. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST '96, Hong Kong, July 1-4). ACM, New York, NY, 27-34.
• 42: KIYOKAWA, K., TAKEMURA, H., AND YOKOYA, N. 1997. Manipulation aid for two-handed 3-D designing within a shared virtual environment. In Proceedings of the 7th International Conference on Human-Computing Interaction (HCI International '97, San Francisco, CA, Aug. 24-29). 937-940.
• 43: KRUEGER, M. 1991. Artificial Reality H. Addison-Wesley, Reading, MA.
• 44: Myron W. Krueger, Environmental technology: making the real world virtual, Communications of the ACM, v.36 n.7, p.36-37, July 1993 [doi>10.1145/159544.159563]
• 45: Myron W. Krueger , Thomas Gionfriddo , Katrin Hinrichsen, VIDEOPLACE—an artificial reality, Proceedings of the SIGCHI conference on Human factors in computing systems, p.35-40, April 1985, San Francisco, California, United States
• 46: Gordon Kurtenbach , George Fitzmaurice , Thomas Baudel , Bill Buxton, The design of a GUI paradigm based on tablets, two-hands, and transparency, Proceedings of the SIGCHI conference on Human factors in computing systems, p.35-42, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258574]
• 47: LEBLANC, A., KALRA, P., MAGNENAT-THALMANN, N., AND THALMANN, D. 1991. Sculpting with the "Ball and mouse" metaphor. In Proceedings of Graphics Interface '91 (Calgary, Alberta, June 3-7), W. A. Davis and B. Wyvill, Eds. Morgan Kaufmann Publishers Inc., San Francisco, CA, 152-159.
• 48: Andrea Leganchuk , Shumin Zhai , William Buxton, Manual and cognitive benefits of two-handed input: an experimental study, ACM Transactions on Computer-Human Interaction (TOCHI), v.5 n.4, p.326-359, Dec. 1998 [doi>10.1145/300520.300522]
• 49: LIANG, J. AND GREEN, M. 1993. JDCAD: A highly interactive 3D modeling system. In Proceedings of the 3rd International Conference on CAD and Computer Graphics (Beijing, China, Aug.). 217-222.
• 50: MACKENZIE, I. S. 1992. Fitts' law as a research and design tool in human-computer interaction. Human-Comput. Interact. 7, 91-139.
• 51: MAPES, D. AND MOSHELL, J. M. 1995. A two-handed interface for object manipulation in virtual environments. Presence: Teleoper. Virtual Environ. 4, 4 (Fall), 403-416.
• 52: MARTENIUK, R., MACKENZIE, C., AND BABA, D. 1984. Bimanual movement control: Information processing and interaction effects. Q. J. Exp. Psychol. 36A, 335-365.
• 53: Nobuyuki Matsushita , Jun Rekimoto, HoloWall: designing a finger, hand, body, and object sensitive wall, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.209-210, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263549]
• 54: MULTIGEN. 1997. MultiGen Smart Scene. MultiGen, Inc., San Jose, CA. http://www.multigen.com/
• 55: Jakob Nielsen, Noncommand user interfaces, Communications of the ACM, v.36 n.4, p.83-99, April 1993 [doi>10.1145/255950.153582]
• 56: Jakob Nielsen, Usability Engineering, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1993
• 57: PETERS, M. 1985. Constraints in the performance of bimanual tasks and their expression in unskilled and skilled subjects. Q. J. Exp. Psychol. 37A, 171-196.
• 58: Timothy Poston , Luis Serra, Dextrous virtual work, Communications of the ACM, v.39 n.5, p.37-45, May 1996 [doi>10.1145/229459.229464]
• 59: POULTON, E. AND FREEMAN, P. 1966. Unwanted asymmetrical transfer effects. Psychol. Bull. 66, 1, 1-8.
• 60: PROVINS, K. AND GLENCROSS, D. 1968. Handwriting, typewriting and handedness. Q. J. Exp. Psychol. 20, 282-289.
• 61: Emanuel Sachs , Andrew Roberts , David Stoops, 3-Draw: A Tool for Designing 3D Shapes, IEEE Computer Graphics and Applications, v.11 n.6, p.18-26, November 1991 [doi>10.1109/38.103389]
• 62: Luis Serra , Ng Hern , Chua Beng Choon , Timothy Poston, Interactive vessel tracing in volume data, Proceedings of the 1997 symposium on Interactive 3D graphics, p.131-ff., April 27-30, 1997, Providence, Rhode Island, United States [doi>10.1145/253284.253320]
• 63: Chris Shaw , Mark Green, Two-handed polygonal surface design, Proceedings of the 7th annual ACM symposium on User interface software and technology, p.205-212, November 02-04, 1994, Marina del Rey, California, United States [doi>10.1145/192426.197517]
• 64: STASSEN, H. AND SMETS, G. 1995. Telemanipulation and telepresence. In Proceedings of the 6th IFAC/IFIP/IFORS/IEA Symposium on Analysis, Design, and Evaluation of Man- Machine Systems (Cambridge, MA, June 27-29). 13-23.
• 65: Richard Stoakley , Matthew J. Conway , Randy Pausch, Virtual reality on a WIM: interactive worlds in miniature, Proceedings of the SIGCHI conference on Human factors in computing systems, p.265-272, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223938]
• 66: D. J. Sturman , D. Zeltzer , S. Pieper, Hands-on interaction with virtual environments, Proceedings of the 2nd annual ACM SIGGRAPH symposium on User interface software and technology, p.19-24, November 13-15, 1989, Williamsburg, Virginia, United States [doi>10.1145/73660.73663]
• 67: Brygg Ullmer , Hiroshi Ishii, The metaDESK: models and prototypes for tangible user interfaces, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.223-232, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263551]
• 68: D. Weimer , S. K. Ganapathy, A synthetic visual environment with hand gesturing and voice input, Proceedings of the SIGCHI conference on Human factors in computing systems: Wings for the mind, p.235-240, March 1989
• 69: WEISER, M. 1991. The computer for the 21st century. Sci. Am. (Sept.), 94-104.
• 70: WING, A. 1982. Timing and co-ordination of repetitive bimanual movements. Q. J. Exp. Psychol. 34A, 339-348.
• 71: WOLFF, P., HURWITZ, I., AND MOSS, H. 1977. Serial organization of motor skills in the left- and right-handed adults. Neuropsychologia 15, 539-546.
• 72: Christopher Richard Wren , Ali Azarbayejani , Trevor Darrell , Alex Paul Pentland, Pfinder: Real-Time Tracking of the Human Body, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.7, p.780-785, July 1997 [doi>10.1109/34.598236]
• 73: Robert C. Zeleznik , Andrew S. Forsberg , Paul S. Strauss, Two pointer input for 3D interaction, Proceedings of the 1997 symposium on Interactive 3D graphics, p.115-ff., April 27-30, 1997, Providence, Rhode Island, United States [doi>10.1145/253284.253316]
• 74: Robert C. Zeleznik , Kenneth P. Herndon , John F. Hughes, SKETCH: an interface for sketching 3D scenes, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.163-170, August 1996 [doi>10.1145/237170.237238]
• 75: Shumin Zhai , William Buxton , Paul Milgram, The partial-occlusion effect: utilizing semitransparency in 3D human-computer interaction, ACM Transactions on Computer-Human Interaction (TOCHI), v.3 n.3, p.254-284, Sept. 1996 [doi>10.1145/234526.234532]
• 76: Shumin Zhai , Paul Milgram , William Buxton, The influence of muscle groups on performance of multiple degree-of-freedom input, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.308-315, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238534]
• 77: Thomas G. Zimmerman , Jaron Lanier , Chuck Blanchard , Steve Bryson , Young Harvill, A hand gesture interface device, Proceedings of the SIGCHI/GI conference on Human factors in computing systems and graphics interface, p.189-192, April 05-09, 1987, Toronto, Ontario, Canada

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.6 Methodology and Techniques
          Subjects: Interaction techniques

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

General Terms:
Design, Experimentation, Human Factors, Measurement

Keywords:
bimanual asymmetry, haptic input, input devices, three-dimensional interaction, two-handed interaction, virtual manipulation