Resolving occlusion in augmented reality

ABSTRACT

Current state-of-the-art augmented reality systems simply overlay computer-generated visuals on the real-world imagery, for example via video or optical see-through displays. However, overlays are not effective when displaying data in three dimensions, since occlusion between the real and computer-generated objects is not addressed.We present a video see-through augmented reality system capable of resolving occlusion between real and computer-generated objects. The heart of our system is a new algorithm that assigns depth values to each pixel in a pair of stereo video images in near-real-time. The algorithm belongs to the class of stereo matching algorithms and thus works in fully dynamic environments. We describe our system in general and the stereo matching algorithm in particular.

REFERENCES

• 1: Arthur, Kevin. Private Communications (August 1994).
• 2: Ronald Azuma , Gary Bishop, Improving static and dynamic registration in an optical see-through HMD, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.197-204, July 1994 [doi>10.1145/192161.192199]
• 3: Michael Bajura , Henry Fuchs , Ryutarou Ohbuchi, Merging virtual objects with the real world: seeing ultrasound imagery within the patient, Proceedings of the 19th annual conference on Computer graphics and interactive techniques, p.203-210, July 1992
• 4: Bajura, Mike and Uhich Neumann. An Improved Mode'l for Augmented Reality Systems. Technical Report TR-94-022, University of North Carolina at Chapel .Hill, Department of Computer Science, Chapel Hill, NC, 1994.
• 5: Stephen T. Barnard , Martin A. Fischler, Computational Stereo, ACM Computing Surveys (CSUR), v.14 n.4, p.553-572, Dec. 1982 [doi>10.1145/356893.356896]
• 6: Carlbom, Ingrid, William Freeman, Gudrun Klinker. Wiliam E. L.orensen. Richard Szeliski. Dtmetri TerLopoulos, and Keith Waters. Computer Vision for Computer Graphics. Couc;e Notes of Course 03 of SIGGRAPH '94 (Orlando. Florida, July 24-29, 1994).
• 7: Caudell, Thomas P. and David W. Mizell. Augmented Reality: An Application of Heads-Up Display Technology to Ma.nual Manufacturing Processes. HICSS, pages 659-669,1992.
• 8: Dhond, Umesb R. and J. K. Aggarwal. Structure from Stereo - A Review. IEEE Transactions on Systetns, Man, and Cybernetics., 19(6):1489-1510, November 1989.
• 9: Steven Feiner , Blair Macintyre , Dorée Seligmann, Knowledge-based augmented reality, Communications of the ACM, v.36 n.7, p.53-62, July 1993 [doi>10.1145/159544.159587]
• 10: Henry Fuchs , Gary Bishop , Kevin Arthur , Leonard McMillan , Ruzena Bajcsy , Sang Lee , Hany Farid , Takeo Kanade, Virtual Space Teleconferencing Using a Sea of Cameras, University of North Carolina at Chapel Hill, Chapel Hill, NC, 1994
• 11: Kanade. "Takeo. Development of a Video-Rate Stereo Machine. In Proceedings of '94 ARPA Itmage Underslatxiing Workshop, November 1994.
• 12: Koch, Reinhard. Automatic Reconstruction of Buildings from Stereoscopic Image Sequences. In R. J. Hubbold and R. Juan, editors, Eurographics '93, pages 339-350, Oxford, UK, 1993. Eurographics, Blackwall Publishers.
• 13: Larry Matthies, Stereo vision for planetary rovers: stochastic modeling to near real-time implementation, International Journal of Computer Vision, v.8 n.1, p.71-91, July 1992 [doi>10.1007/BF00126401]
• 14: Nishihara., H. K. Real-Time Implementation of a Sign- Correlation Algorithm for Image-Matching. Technical Report 90-2, Telcos Research, February 1990.
• 15: Ross. Bill. A Practical Stereo Vision System. III Proceedings of Cornpurer Vision and Patrem Recognifion '93, 1993.
• 16: Stereograpbics. CrystalEyes Video System. 1994.
• 17: Wloka, hlatthias M. Lag in Multiprocessor Virtual Reality. Presence, 4( 1). 1994. To appear.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Virtual reality

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Stereo

  J. Computer Applications
  J.7 COMPUTERS IN OTHER SYSTEMS
      Subjects: Real time

General Terms:
Algorithms

Keywords:
approximation, augmented reality, dynamic environments, interaction, occlusion, real-time, stereo matching