Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

Using multiple cameras, a dynamic real world scene can be captured, represented, and streamed for realistic interaction in a 3D immersive system. One method to represent the dynamic environment is to use multiple depth streams - streams with color and depth information. However the captured data is too large to be streamed uncompressed. Fortunately, these data exhibit spatial and temporal coherence that can be used for compression.

In this paper, we examine the initial step in compressing multiple depth streams. To use spatial coherence between streams to compress multiple depth streams, base reference streams - intrastreams - have to be encoded without the use of other depth streams. We investigate encoding of this base depth stream using only temporal coherence. The proposed algorithm extends video stream encoding techniques to encode color and depth. We show that for encoding a depth stream which requires high quality, such as base reference streams, using separate motion vectors to encode color and depth performs better than using a single motion vector.

REFERENCES

• 1: 3D Telepresence for Medical Consultation. http://www.cs.unc.edu/Research/nlm.
• 2: H. H. Baker et al. The Coliseum immersive teleconferencing system. In ITP 2002, Juan Les Pins, France, Dec. 2002.
• 3: Canesta Vision. http://www.canesta.com.
• 4: Jin-Xiang Chai , Xin Tong , Shing-Chow Chan , Heung-Yeung Shum, Plenoptic sampling, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.307-318, July 2000 [doi>10.1145/344779.344932]
• 5: Chun-Fa Chang , Gary Bishop , Anselmo Lastra, LDI tree: a hierarchical representation for image-based rendering, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.291-298, July 1999 [doi>10.1145/311535.311571]
• 6: C. Fehn et al. An evolutionary and optimised approach on 3D-TV. In International Broadcast Conference, pages 357--365, Amsterdam, The Netherlands, Sept. 2002.
• 7: Markus Gross , Stephan Würmlin , Martin Naef , Edouard Lamboray , Christian Spagno , Andreas Kunz , Esther Koller-Meier , Tomas Svoboda , Luc Van Gool , Silke Lang , Kai Strehlke , Andrew Vande Moere , Oliver Staadt, blue-c: a spatially immersive display and 3D video portal for telepresence, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003 [doi>10.1145/882262.882350]
• 8: ISO/IEC JTC1/SC29/WG11. Information Technology - Generic Coding of Moving Pictures and Assoicated Audio Information: Video. ISO/IEC 13818-2, 1992.
• 9: ISO/IEC JTC1/SC29/WG11. Information Technology - Coding of Audio-Visual Objects - Part2: Visual. ISO/IEC 14496-2, 2004.
• 10: ISO/IEC JTC1/SC29/WG11. Information Technology - Coding of Audio-Visual Objects - Part10: Advanced Video Coding. ISO/IEC 14496-10, 2005.
• 11: ISO/IEC JTC1/SC29/WG11. Information Technology - Coding of Audio-Visual Objects - Part16: Animation Framework eXtension. ISO/IEC 14496-16, 2005.
• 12: Peter Kauff , Oliver Schreer, An immersive 3D video-conferencing system using shared virtual team user environments, Proceedings of the 4th international conference on Collaborative virtual environments, p.105-112, September 30-October 02, 2002, Bonn, Germany [doi>10.1145/571878.571895]
• 13: N. Kelshikar et al. Real-time terascale implementation of tele-immersion. In the Terascale Performance Analysis Workshop, Melbourne, Australia, June 2003.
• 14: Sang-Uok Kum , Ketan Mayer-Patel, Real-time multidepth stream compression, ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP), v.1 n.2, p.128-150, May 2005 [doi>10.1145/1062253.1062255]
• 15: Sang-Uok Kum , Ketan Mayer-Patel , Henry Fuchs, Real-time compression for dynamic 3D environments, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957050]
• 16: Marc Levoy , Pat Hanrahan, Light field rendering, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.31-42, August 1996 [doi>10.1145/237170.237199]
• 17: M. Magnor and B. Girod. Data compression in image-based rendering. IEEE Trans. on Circuits and Systems for Video Technology, 10(3):338--343, Apr. 2000.
• 18: Wojciech Matusik , Chris Buehler , Ramesh Raskar , Steven J. Gortler , Leonard McMillan, Image-based visual hulls, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.369-374, July 2000 [doi>10.1145/344779.344951]
• 19: Wojciech Matusik , Hanspeter Pfister, 3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004 [doi>10.1145/1015706.1015805]
• 20: Jonathan Shade , Steven Gortler , Li-wei He , Richard Szeliski, Layered depth images, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.231-242, July 1998 [doi>10.1145/280814.280882]
• 21: A. Smolic and P. Kauff. Interactive 3-D video representation and coding technologies. Proc. of the IEEE, 93(1):98--110, Jan. 2005.
• 22: H. Towles et al. 3D tele-immersion over internet2. In ITP 2002, Juan Les Pins, France, Dec. 2002.
• 23: S. Würmlin, E. Lamboray, and M. Gross. 3D video fragments: Dynamic point samples for real-time free-viewpoint video. Computers & Graphics, 28(1):3--14, Feb. 2004.
• 24: XviD. http://www.xvid.org.
• 25: ZCam Depth Camera. http://www.3dvsystems.com.
• 26: C. Lawrence Zitnick , Sing Bing Kang , Matthew Uyttendaele , Simon Winder , Richard Szeliski, High-quality video view interpolation using a layered representation, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004 [doi>10.1145/1015706.1015766]