VirtualizeMe: interactive model reconstruction from stereo video streams

ABSTRACT

Tele-Immersion depends on face-to-face, viewpoint corrected, stereoscopic, virtual environments, allowing users to naturally interact with each other and the digital environment surrounding them, via realistic avatars. This paper presents an extraction, modeling and communication methodology for the creation of the needed avatars, based on a scalable focused disparity map (FDM) approach. The FDM enables variable distance reconstruction of dynamic target objects despite restrictions in disparity range. Combined with a pixel-based disparity cost interpolation, a sub-pixel disparity refinement is achieved, providing a high depth-resolution and smooth reconstruction of the target objects. This technique approaches real-time characteristics, extracting avatar models and communicating them to remote render nodes at 15 frames per second.

REFERENCES

• 1: Griesser, A., Roeck, D. S., Neubeck, A., and L., V. G. 2005. Gpu-based foreground-background segmentation using an extended colinearity criterion. In Proc. of Vision, Modeling, and Visualization (VMV), 319--326.
• 2: Gross, M., Wrmlin, S., Naef, M., Lamboray, E., Spagno, C., Kunz, A., Koller-Meier, E., Svoboda, T., Gool, L. V., Lang, S., Strehlke, K., Moere, A. V., and Staadt, O. 2003. blue-c: A spatially immersive display and 3d video portal for telepresence. 819--827.
• 3: Rudolf Mester , Til Aach , Lutz Dümbgen, Illumination-Invariant Change Detection Using a Statistical Colinearity Criterion, Proceedings of the 23rd DAGM-Symposium on Pattern Recognition, p.170-177, September 12-14, 2001
• 4: Seon-Min Rhee , Remo Ziegler , Jiyoung Park , Martin Naef , Markus Gross , Myoung-Hee Kim, Low-Cost Telepresence for Collaborative Virtual Environments, IEEE Transactions on Visualization and Computer Graphics, v.13 n.1, p.156-166, January 2007 [doi>10.1109/TVCG.2007.17]
• 5: Sadagic, A., H., T., Lanier, J., Fuchs, H., Van Dam, A., Daniilidis, K., Mulligan, J., Holden, L., and Zelenik, 2001. National tele-immersion initiative: Towards compelling tele-immersive collaborative environments. Presentation given at Medicine meets Virtual Reality 2001 Conference.
• 6: Larry L. Smarr , Andrew A. Chien , Tom DeFanti , Jason Leigh , Philip M. Papadopoulos, The OptIPuter, Communications of the ACM, v.46 n.11, November 2003 [doi>10.1145/948383.948410]
• 7: Wanmin Wu , Zhenyu Yang , Klara Nahrstedt , Gregorij Kurillo , Ruzena Bajcsy, Towards multi-site collaboration in tele-immersive environments, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291406]
• 8: Yang, Q., Wang, L., Yang, R., Wang, S., Liao, M., and Nistér, D. 2006. Real-time global stereo matching using hierarchical belief propagation. In BMVC, British Machine Vision Association, 989--998.
• 9: Yang, Q., Yang, R., Davis, J., and Nistér, D. 2007. Spatial-depth super resolution for range images. In CVPR, IEEE Computer Society, Minneapolis, MN, USA, 1--8.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.3 Communications Applications
          Subjects: Computer conferencing, teleconferencing, and videoconferencing

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Virtual reality
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Stereo

Keywords:
3D from video, background extraction, disparity maps, point cloud streaming, tele-immersion