Semantic pointing for object picking in complex 3D environments

ABSTRACT

Today's large and high-resolution displays coupled with powerful graphics hardware offer the potential for highly realistic 3D virtual environments, but also cause increased target acquisition difficulty for users interacting with these environments. We present an adaptation of semantic pointing to object picking in 3D environments. Essentially, semantic picking shrinks empty space and expands potential targets on the screen by dynamically adjusting the ratio between movement in visual space and motor space for relative input devices such as the mouse. Our implementation operates in the image-space using a hierarchical representation of the standard stencil buffer to allow for real-time calculation of the closest targets for all positions on the screen. An informal user study indicates that subjects perform more accurate pointing with semantic 3D pointing than without.

REFERENCES

• 1: David Ahlström, Modeling and improving selection in cascading pull-down menus using Fitts' law, the steering law and force fields, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1054982]
• 2: C. Andujar and F. Argelaguet. Virtual pads: Decoupling motor space and visual space for flexible manipulation of 2D windows within VEs. In Proceedings of the IEEE Symposium on 3D User Interfaces, pages 99--106, 2007.
• 3: Ravin Balakrishnan, "Beating" Fitts' law: virtual enhancements for pointing facilitation, International Journal of Human-Computer Studies, v.61 n.6, p.857-874, December 2004 [doi>10.1016/j.ijhcs.2004.09.002]
• 4: P. Baudisch, E. Cutrell, M. Czerwinski, D. C. Robbins, P. Tandler, B. B. Bederson, and A. Zierlinger. Drag-and-pop and drag-and-pick: Techniques for accessing remote screen content on touch- and penoperated systems. In Proceedings of INTERACT, pages 57--64, 2003.
• 5: Patrick Baudisch , Edward Cutrell , Ken Hinckley , Adam Eversole, Snap-and-go: helping users align objects without the modality of traditional snapping, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055014]
• 6: Benjamin B. Bederson, Fisheye menus, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.217-225, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354782]
• 7: Renaud Blanch , Yves Guiard , Michel Beaudouin-Lafon, Semantic pointing: improving target acquisition with control-display ratio adaptation, Proceedings of the SIGCHI conference on Human factors in computing systems, p.519-526, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985758]
• 8: Richard A. Bolt, “Put-that-there”: Voice and gesture at the graphics interface, Proceedings of the 7th annual conference on Computer graphics and interactive techniques, p.262-270, July 14-18, 1980, Seattle, Washington, United States
• 9: Doug A. Bowman , Larry F. Hodges, An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments, Proceedings of the 1997 symposium on Interactive 3D graphics, p.35-ff., April 27-30, 1997, Providence, Rhode Island, United States [doi>10.1145/253284.253301]
• 10: Andy Cockburn , Philip Brock, Human on-line response to visual and motor target expansion, Proceedings of Graphics Interface 2006, June 07-09, 2006, Quebec, Canada
• 11: A. Cockburn and A. Firth. Improving the acquisition of small targets. In Proceedings of the British HCI Conference, pages 181--196, 2003.
• 12: P. M. Fitts. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47:381--391, 1954.
• 13: G. W. Furnas, Generalized fisheye views, Proceedings of the SIGCHI conference on Human factors in computing systems, p.16-23, April 13-17, 1986, Boston, Massachusetts, United States
• 14: Tovi Grossman , Ravin Balakrishnan, The bubble cursor: enhancing target acquisition by dynamic resizing of the cursor's activation area, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055012]
• 15: Yves Guiard , Renaud Blanch , Michel Beaudouin-Lafon, Object pointing: a complement to bitmap pointing in GUIs, Proceedings of Graphics Interface 2004, p.9-16, May 17-19, 2004, London, Ontario, Canada
• 16: Carl Gutwin, Improving focus targeting in interactive fisheye views, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503424]
• 17: Paul Kabbash , William A. S. Buxton, The “prince” technique: Fitts' law and selection using area cursors, Proceedings of the SIGCHI conference on Human factors in computing systems, p.273-279, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223939]
• 18: S. MacKenzie. Fitts' law as a research and design tool in human-computer interaction. Human-Computer Interaction, 7:91--139, 1992.
• 19: S. Mackenzie and S. Riddersma. Effects of output display and control-display gain on human performance in interactive systems, Behaviour and Information Technology, 13(5):328--337, 1994.
• 20: Michael McGuffin , Ravin Balakrishnan, Acquisition of expanding targets, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503388]
• 21: Jeffrey S. Pierce , Brian C. Stearns , Randy Pausch, Voodoo dolls: seamless interaction at multiple scales in virtual environments, Proceedings of the 1999 symposium on Interactive 3D graphics, p.141-145, April 26-29, 1999, Atlanta, Georgia, United States [doi>10.1145/300523.300540]
• 22: Ivan Poupyrev , Mark Billinghurst , Suzanne Weghorst , Tadao Ichikawa, The go-go interaction technique: non-linear mapping for direct manipulation in VR, Proceedings of the 9th annual ACM symposium on User interface software and technology, p.79-80, November 06-08, 1996, Seattle, Washington, United States [doi>10.1145/237091.237102]
• 23: I. Poupyrev, S. Weghorst, M. Billinghurst, and T. Ichikawa. Egocentric object manipulation in virtual environments: Empirical evaluation of interaction techniques. In Proceedings of Eurographics, 1998.
• 24: Frank Steinicke , Klaus Hinrichs, Grab-and-Throw Metaphor: Adapting Desktop-based Interaction Paradigms to Virtual Reality, Proceedings of the 3D User Interfaces, p.83-86, March 25-29, 2006
• 25: Richard Stoakley , Matthew J. Conway , Randy Pausch, Virtual reality on a WIM: interactive worlds in miniature, Proceedings of the SIGCHI conference on Human factors in computing systems, p.265-272, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223938]
• 26: L. Vanacken, T. Grossman, and K. Coninx. Exploring the effects of environment density and target visibility on object selection in 3D virtual environments. In Proceedings of the IEEE Symposium on 3D User Interfaces, pages 115--122, 2007.
• 27: Aileen Worden , Nef Walker , Krishna Bharat , Scott Hudson, Making computers easier for older adults to use: area cursors and sticky icons, Proceedings of the SIGCHI conference on Human factors in computing systems, p.266-271, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258724]
• 28: Shumin Zhai , Stéphane Conversy , Michel Beaudouin-Lafon , Yves Guiard, Human on-line response to target expansion, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642644]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism

Keywords:
3D picking, Fitts' law, motor space, target acquisition, visual space