View management for virtual and augmented reality

ABSTRACT

We describe a view-management component for interactive 3D user interfaces. By view management, we mean maintaining visual constraints on the projections of objects on the view plane, such as locating related objects near each other, or preventing objects from occluding each other. Our view-management component accomplishes this by modifying selected object properties, including position, size, and transparency, which are tagged to indicate their constraints. For example, some objects may have geometric properties that are determined entirely by a physical simulation and which cannot be modified, while other objects may be annotations whose position and size are flexible.We introduce algorithms that use upright rectangular extents to represent on the view plane a dynamic and efficient approximation of the occupied space containing the projections of visible portions of 3D objects, as well as the unoccupied space in which objects can be placed to avoid occlusion. Layout decisions from previous frames are taken into account to reduce visual discontinuities. We present augmented reality and virtual reality examples to which we have applied our approach, including a dynamically labeled and annotated environment.

REFERENCES

• 1: Christopher Ahlberg , Christopher Williamson , Ben Shneiderman, Dynamic queries for information exploration: an implementation and evaluation, Proceedings of the SIGCHI conference on Human factors in computing systems, p.619-626, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.143054]
• 2: K. Arthur, T. Preston, R. Taylor II, F. Brooks, Jr., M. Whitton, and W. Wright. Designing and building the PIT: A head tracked stereo workspace for two users. In Second Int. Immersive Projection Technology Workshop, Ames, IA, May 11-12 1998. www.cs.unc.edu/Research/graphics/ GRIP/PIT/doc/ipt-paper.pdf.
• 3: G. J. Badros, J. Nichols, and A. Boming. SCWM-an intelligent constraint-enabled window manager. In Proc. AAAI Spring Symposium on Smart Graphics, Cambridge, MA, Mar.20-22 2000. (http://scwm.mit.edu).
• 4: Giuseppe Di Battista , Peter Eades , Roberto Tamassia , Ioannis G. Tollis, Graph Drawing: Algorithms for the Visualization of Graphs, Prentice Hall PTR, Upper Saddle River, NJ, 1998
• 5: Blaine A. Bell , Steven K. Feiner, Dynamic space management for user interfaces, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.239-248, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354790]
• 6: M. Billinghurst, S. Weghorst, and T. Fumess III. Shared space: An augmented reality approach for computer supported collaborative work. Virtual Reality, 3(1):25-36, 1998.
• 7: Andreas Butz , Tobias Höllerer , Steven Feiner , Blair MacIntyre , Clifford Beshers, Enveloping Users and Computers in a Collaborative 3D Augmented Reality, Proceedings of the 2nd IEEE and ACM International Workshop on Augmented Reality, p.35, October 20-21, 1999
• 8: M. Sheelagh T. Carpendale , David J. Cowperthwaite , F. David Fracchia, Extending Distortion Viewing from 2D to 3D, IEEE Computer Graphics and Applications, v.17 n.4, p.42-51, July 1997 [doi>10.1109/38.595268]
• 9: Jon Christensen , Joe Marks , Stuart Shieber, An empirical study of algorithms for point-feature label placement, ACM Transactions on Graphics (TOG), v.14 n.3, p.203-232, July 1995 [doi>10.1145/212332.212334]
• 10: Y. Chrysanthou, and M. Slater. Computing dynamic changes to BSP trees, Computer Graphics Forum (Proc. Eurographics '92), 11(3), September 1992, 321-332.
• 11: Ellis S Cohen , Edward T Smith , Lee A Iverson, Constraint-based tile windows, IEEE Computer Graphics and Applications, v.6 n.5, p.35-45, May 1986 [doi>10.1109/MCG.1986.276790]
• 12: M. Deering and H. Sowizral. Java3D Specification, Version 1.0. Sun Microsystems, 2550 Garcia Avenue, Mountain View, CA 94043, USA, Aug. 1997.
• 13: K. Fairchild, S. Poltrock, and G. Furnas. SemNet: Threedimensional graphic representations of large knowledge bases. In R. Guindon, editor, Cognitive Science and its Applications for Human Computer Interaction, pages 201- 233. Lawrence Erlbaum, Hillsdale, NJ, 1988.
• 14: Steven Feiner , Blair MacIntyre , Marcus Haupt , Eliot Solomon, Windows on the world: 2D windows for 3D augmented reality, Proceedings of the 6th annual ACM symposium on User interface software and technology, p.145-155, December 1993, Atlanta, Georgia, United States [doi>10.1145/168642.168657]
• 15: Steven Feiner , Blair Macintyre , Dorée Seligmann, Knowledge-based augmented reality, Communications of the ACM, v.36 n.7, p.53-62, July 1993 [doi>10.1145/159544.159587]
• 16: S. Feiner and D. Seligmann. Cutaways and ghosting: Satisfying visibility constraints in dynamic 3D illustrations. The Visual Computer, 8(5-6):292-302, June 1992.
• 17: James D. Foley , Andries van Dam , Steven K. Feiner , John F. Hughes, Computer graphics (2nd ed. in C): principles and practice, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1995
• 18: Henry Fuchs , Zvi M. Kedem , Bruce F. Naylor, On visible surface generation by a priori tree structures, Proceedings of the 7th annual conference on Computer graphics and interactive techniques, p.124-133, July 14-18, 1980, Seattle, Washington, United States
• 19: Li-wei He , Michael F. Cohen , David H. Salesin, The virtual cinematographer: a paradigm for automatic real-time camera control and directing, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.217-224, August 1996 [doi>10.1145/237170.237259]
• 20: Gerd Hesina , Dieter Schmalstieg , Anton Furhmann , Werner Purgathofer, Distributed Open Inventor: a practical approach to distributed 3D graphics, Proceedings of the ACM symposium on Virtual reality software and technology, p.74-81, December 20-22, 1999, London, United Kingdom [doi>10.1145/323663.323675]
• 21: S. A. Hirsch. An algorithm for automatic name placement around point data. The American Cartographer, 9(1 ): 5-17, 1982.
• 22: E. Imhof. Positioning names on maps. The American Cartographer, 2(2):128-144, 1975.
• 23: S. Julier, M. Lanzagorta, Y. Baillot, L. Rosenblum, S. Feiner, T. H611erer, and S. Sestito. Information filtering for mobile augmented reality. In Proc. 1SAR '00 (Int. Symposium on Augmented Reality), pages 3-11, Munich, Germany, October 5-6 2000.
• 24: Tomihisa Kamada , Satoru Kawai, An enhanced treatment of hidden lines, ACM Transactions on Graphics (TOG), v.6 n.4, p.308-323, Oct. 1987 [doi>10.1145/35039.35042]
• 25: Blair MacIntyre , Steven Feiner, A distributed 3D graphics library, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.361-370, July 1998 [doi>10.1145/280814.280935]
• 26: Mark R. Mine , Frederick P. Brooks, Jr. , Carlo H. Sequin, Moving objects in space: exploiting proprioception in virtual-environment interaction, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.19-26, August 1997 [doi>10.1145/258734.258747]
• 27: T. Mori, K. Koiso, and K. Tanaka. Spatial data presentation by LOD control based on distance, orientation and differentiation. In Proc. UM3 '99 (Int. Workshop on Urban 3D/Multimedia Mapping), pages 49-56, Tokyo, Japan, 1999.
• 28: Cary B. Phillips , Norman I. Badler , John Granieri, Automatic viewing control for 3D direct manipulation, Proceedings of the 1992 symposium on Interactive 3D graphics, p.71-74, June 1992, Cambridge, Massachusetts, United States [doi>10.1145/147156.147167]
• 29: George G. Robertson , Stuart K. Card, Fix and float: object movement by egocentric navigation, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.149-150, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263535]
• 30: Hanan Samet, The design and analysis of spatial data structures, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1990
• 31: T. Stamer, S. Mann, B. Rhodes, J. Levine, J. Healey, D. Kirsch, R. Picard, and A. Pentland. Augmented reality through wearable computing. Presence, 6(4):386-398, August 1997.
• 32: Richard Stoakley , Matthew J. Conway , Randy Pausch, Virtual reality on a WIM: interactive worlds in miniature, Proceedings of the SIGCHI conference on Human factors in computing systems, p.265-272, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223938]
• 33: I. Sutherland. A head-mounted three dimensional display. In Proc. FJCC 1968, pages 757-764, Washington, DC, 1968. Thompson Books.
• 34: Z. Szalavari, D. Schmalstieg, A. Fuhrmann, and M. Gervautz. Studierstube: An environment for collaboration \ in augmented, reahty. Virtual Reality, 3(1):37-48, 1998.
• 35: W. Teitelman. A tour through CEDAR. 1EEE Software, 1(2):44-73, April 1984.
• 36: William C. Thibault , Bruce F. Naylor, Set operations on polyhedra using binary space partitioning trees, ACM SIGGRAPH Computer Graphics, v.21 n.4, p.153-162, July 1987
• 37: J. van Roessel. An algorithm for locating candidate labeling boxes within a polygon. The American Cartographer, 16(3):201-209, 1989.
• 38: Suya You , Ulrich Neumann , Ronald Azuma, Hybrid Inertial and Vision Tracking for Augmented Reality Registration, Proceedings of the IEEE Virtual Reality, p.260, March 13-17, 1999

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Artificial, augmented, and virtual realities

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Screen design (e.g., text, graphics, color); Graphical user interfaces (GUI)

  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.6 Methodology and Techniques
          Subjects: Interaction techniques
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Virtual reality

Keywords:
annotation, augmented reality, environment management, labeling, view management, virtual environments, wearable computing