blue-c: a spatially immersive display and 3D video portal for telepresence

ABSTRACT

We present blue-c, a new immersive projection and 3D video acquisition environment for virtual design and collaboration. It combines simultaneous acquisition of multiple live video streams with advanced 3D projection technology in a CAVE™-like environment, creating the impression of total immersion. The blue-c portal currently consists of three rectangular projection screens that are built from glass panels containing liquid crystal layers. These screens can be switched from a whitish opaque state (for projection) to a transparent state (for acquisition), which allows the video cameras to "look through" the walls. Our projection technology is based on active stereo using two LCD projectors per screen. The projectors are synchronously shuttered along with the screens, the stereo glasses, active illumination devices, and the acquisition hardware. From multiple video streams, we compute a 3D video representation of the user in real time. The resulting video inlays are integrated into a networked virtual environment. Our design is highly scalable, enabling blue-c to connect to portals with less sophisticated hardware.

REFERENCES

• 1: Maneesh Agrawala , Andrew C. Beers , Ian McDowall , Bernd Fröhlich , Mark Bolas , Pat Hanrahan, The two-user Responsive Workbench: support for collaboration through individual views of a shared space, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.327-332, August 1997 [doi>10.1145/258734.258875]
• 2: Chris Buehler , Michael Bosse , Leonard McMillan , Steven Gortler , Michael Cohen, Unstructured lumigraph rendering, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.425-432, August 2001 [doi>10.1145/383259.383309]
• 3: Carolina Cruz-Neira , Daniel J. Sandin , Thomas A. DeFanti, Surround-screen projection-based virtual reality: the design and implementation of the CAVE, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.135-142, September 1993 [doi>10.1145/166117.166134]
• 4: DAVIS, J. W. AND BOBICK, A. F. 1998. Sideshow: A silhouette-based interactive dual-screen environment. Technical Report 457, MIT Media Lab.
• 5: Paul Debevec , Andreas Wenger , Chris Tchou , Andrew Gardner , Jamie Waese , Tim Hawkins, A lighting reproduction approach to live-action compositing, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 6: Simon J. Gibbs , Constantin Arapis , Christian J. Breiteneder, TELEPORT— towards immersive copresence, Multimedia Systems, v.7 n.3, p.214-221, May 1999 [doi>10.1007/s005300050123]
• 7: Peter Kauff , Oliver Schreer, An immersive 3D video-conferencing system using shared virtual team user environments, Proceedings of the 4th international conference on Collaborative virtual environments, p.105-112, September 30-October 02, 2002, Bonn, Germany [doi>10.1145/571878.571895]
• 8: Wolfgang Krüger , Christian-A. Bohn , Bernd Fröhlich , Heinrich Schüth , Wolfgang Strauss , Gerold Wesche, The Responsive Workbench: A Virtual Work Environment, Computer, v.28 n.7, p.42-48, July 1995 [doi>10.1109/2.391040]
• 9: LAMPERT, C. 1999. The world of large-area glazing and displays. In Switchable Materials and Flat Panel Displays, pages 2--11. SPIE.
• 10: A. Laurentini, The Visual Hull Concept for Silhouette-Based Image Understanding, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.16 n.2, p.150-162, February 1994 [doi>10.1109/34.273735]
• 11: Wojciech Matusik , Chris Buehler , Leonard McMillan, Polyhedral Visual Hulls for Real-Time Rendering, Proceedings of the 12th Eurographics Workshop on Rendering Techniques, p.115-126, June 25-27, 2001
• 12: Wojciech Matusik , Chris Buehler , Ramesh Raskar , Steven J. Gortler , Leonard McMillan, Image-based visual hulls, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.369-374, July 2000 [doi>10.1145/344779.344951]
• 13: Rudolf Mester , Til Aach , Lutz Dümbgen, Illumination-Invariant Change Detection Using a Statistical Colinearity Criterion, Proceedings of the 23rd DAGM-Symposium on Pattern Recognition, p.170-177, September 12-14, 2001
• 14: Saied Moezzi , Arun Katkere , Don Y. Kuramura , Ramesh Jain, Immersive Video, Proceedings of the 1996 Virtual Reality Annual International Symposium (VRAIS 96), p.17, March 30-April 03, 1996
• 15: MULLIGAN, J. AND DANIILIDIS, K. 2000. View-independent scene acquisition for tele-presence. In Proceedings of the International Symposium on Augmented Reality, pages 105--110. IEEE Computer Society Press.
• 16: Martin Naef , Edouard Lamboray , Oliver Staadt , Markus Gross, The blue-c distributed scene graph, Proceedings of the workshop on Virtual environments 2003, p.125-133, May 22-23, 2003, Zurich, Switzerland [doi>10.1145/769953.769968]
• 17: Martin Naef , Oliver Staadt , Markus Gross, Spatialized audio rendering for immersive virtual environments, Proceedings of the ACM symposium on Virtual reality software and technology, November 11-13, 2002, Hong Kong, China [doi>10.1145/585740.585752]
• 18: P. J. Narayanan , Peter W. Rander , Takeo Kanade, Constructing Virtual Worlds Using Dense Stereo, Proceedings of the Sixth International Conference on Computer Vision, p.3, January 04-07, 1998
• 19: Ken Perlin , Salvatore Paxia , Joel S. Kollin, An autostereoscopic display, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.319-326, July 2000 [doi>10.1145/344779.344933]
• 20: Stephen Pollard , Sean Hayes, View synthesis by edge transfer with application to the generation of immersive video objects, Proceedings of the ACM symposium on Virtual reality software and technology, p.91-98, November 02-05, 1998, Taipei, Taiwan [doi>10.1145/293701.293713]
• 21: Marc Pollefeys , Reinhard Koch , Luc Van Gool, Self-Calibration and Metric Reconstruction Inspite of Varying and Unknown Intrinsic Camera Parameters, International Journal of Computer Vision, v.32 n.1, p.7-25, Aug. 1. 1999 [doi>10.1023/A:1008109111715]
• 22: Ramesh Raskar , Greg Welch , Matt Cutts , Adam Lake , Lev Stesin , Henry Fuchs, The office of the future: a unified approach to image-based modeling and spatially immersive displays, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.179-188, July 1998 [doi>10.1145/280814.280861]
• 23: Erik Reinhard , Michael Ashikhmin , Bruce Gooch , Peter Shirley, Color Transfer between Images, IEEE Computer Graphics and Applications, v.21 n.5, p.34-41, September 2001
• 24: John Rohlf , James Helman, IRIS performer: a high performance multiprocessing toolkit for real-time 3D graphics, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.381-394, July 1994 [doi>10.1145/192161.192262]
• 25: SADAGIC, A., TOWLES, H., LANIER, J., FUCHS, H., VAN DAM, A., DANIILIDIS, K., MULLIGAN, J., HOLDEN, L., AND ZELEZNIK, B. 2001. National tele-immersion initiative: Towards compelling tele-immersive collaborative environments. Presentation given at Medicine meets Virtual Reality 2001 Conference.
• 26: SCHMIDT, D.. Real-time CORBA with TAO (The ACE ORB). http://www.cs.wustl.edu/~schmidt/TAO.html.
• 27: Christian P. Spagno , Andreas M. Kunz, Construction of a Three-sided Immersive Telecollaboration System, Proceedings of the IEEE Virtual Reality 2003, p.37, March 22-26, 2003
• 28: SUBRAMANIAN, S., RAJAN, V., KEENAN, D., SANDIN, D., DEFANTI, T., AND JOHNSON, A. 2002. A realistic video avatar system for networked virtual environments. In Proceedings of Seventh Annual Immersive Projection Technology Symposium.
• 29: Sundar Vedula , Simon Baker , Takeo Kanade, Spatio-temporal view interpolation, Proceedings of the 13th Eurographics workshop on Rendering, June 26-28, 2002, Pisa, Italy
• 30: Stephan Würmlin , Edouard Lamboray , Oliver G. Staadt , Markus H. Gross, 3D Video Recorder, Proceedings of the 10th Pacific Conference on Computer Graphics and Applications, p.325, October 09-11, 2002
• 31: WÜRMLIN, S., LAMBORAY, E., AND GROSS, M. H. 2003. 3D video fragments: Dynamic point samples for real-time free-viewpoint video. Technical Report No. 397, Institute of Scientific Computing, ETH Zurich.
• 32: Matthias Zwicker , Hanspeter Pfister , Jeroen van Baar , Markus Gross, Surface splatting, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.371-378, August 2001 [doi>10.1145/383259.383300]

INDEX TERMS

Keywords:
3D Video, graphics hardware, real-time graphics, spatially immersive displays, virtual environments