A framework of individually-focused teleconferencing (IFT) via an efficient 3D reprojection technique

ABSTRACT

In this paper, we propose a framework on individually-focused teleconferencing (IFT) which is supported by an efficient 3D reprojection technique. The goal of designing IFT is to provide sufficient 3D pose information to each teleconference participant in order to establish lively communication in a teleconference on low-bandwidth internet. A novel IFT deployment and an efficient 3D reprojection technique are two major contributions of this paper. Our 3D reprojection technique uses a mirror reference view in three-view epipolar geometry. It overcomes the inefficiency in previous techniques and provides visually good recovery of pose information of teleconference participants even though it is theoretically an approximation scheme.

REFERENCES

• 1: S. Avidan , A. Shashua, Novel view synthesis in tensor space, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.1034, June 17-19, 1997
• 2: Shai Avidan , Amnon Shashua, Novel View Synthesis by Cascading Trilinear Tensors, IEEE Transactions on Visualization and Computer Graphics, v.4 n.4, p.293-306, October 1998 [doi>10.1109/2945.765324]
• 3: Shenchang Eric Chen , Lance Williams, View interpolation for image synthesis, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.279-288, September 1993 [doi>10.1145/166117.166153]
• 4: S. Deering. Host extensions for ip multicast. In Request For Comment 1112. http://asg.web.cmu.edu/rfc/rfc1112.html, Aug, 1989.
• 5: Joe John Duran, Mainstream Videoconferencing: A Developer's Guide to Distance Multimedia, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1997
• 6: Kishino F. Virtual space teleconferencing system - real time detection and reproduction of human images. In Proc. Imagina 94, pages 109--118, 1994.
• 7: Andrew W. Fitzgibbon , Andrew Zisserman, Automatic Camera Recovery for Closed or Open Image Sequences, Proceedings of the 5th European Conference on Computer Vision-Volume I, p.311-326, June 02-06, 1998
• 8: H. Fuchs, G. Bishop, K. Arthur, L. McMillan, R. Bajcsy, S. Lee, H. Farid, and T. Kanade. Virtual space teleconferencing using a sea of cameras. In Proc. First International Conference on Medical Robotics and Computer Assited Surgery, pages 161--167, 1994.
• 9: Chris Greenhalgh , Steven Benford, MASSIVE: a collaborative virtual environment for teleconferencing, ACM Transactions on Computer-Human Interaction (TOCHI), v.2 n.3, p.239-261, Sept. 1995 [doi>10.1145/210079.210088]
• 10: R. Hartley and P. Sturm. Triangulation. In American Image Understanding, pages 957--966, 1994.
• 11: Richard Hartley , Andrew Zisserman, Multiple view geometry in computer visiond, Cambridge University Press, New York, NY, 2001
• 12: Richard I. Hartley, In Defense of the Eight-Point Algorithm, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.6, p.580-593, June 1997 [doi>10.1109/34.601246]
• 13: Z. Liu, Z. Zhang, C. Jacobs, and M. Cohen. Rapid modeling of animated faces from video. Technical Report MSR-TR-2000-11, Microsoft Cooperation, Feb 2000.
• 14: Michael R. Macedonia , Donald P. Brutzman, MBone Provides Audio and Video Across the Internet, Computer, v.27 n.4, p.30-36, April 1994 [doi>10.1109/2.274996]
• 15: Leonard McMillan , Gary Bishop, Plenoptic modeling: an image-based rendering system, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.39-46, September 1995 [doi>10.1145/218380.218398]
• 16: J. Ohya, Y. Kitamura, H. Takemura, F. Kishino, and N. Terashima. Real-time reproduction of 3d human images in virtual space teleconferencing. In Proceedings of the IEEE Virtual Reality Annual International Symposium, pages 408--414, 1993.
• 17: Frederic Ira Parke, A parametric model for human faces., 1974
• 18: Ramesh Raskar , Greg Welch , Matt Cutts , Adam Lake , Lev Stesin , Henry Fuchs, The office of the future: a unified approach to image-based modeling and spatially immersive displays, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.179-188, July 1998 [doi>10.1145/280814.280861]
• 19: E. Rosen. Personal videoconferencing. Manning, 1996.
• 20: Richard Szeliski , Heung-Yeung Shum, Creating full view panoramic image mosaics and environment maps, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.251-258, August 1997 [doi>10.1145/258734.258861]
• 21: P. Torr and A. Zisserman. Robust parameterization and computation of the trifocal tensor. Image and Vision Computing, 15:591--605, 1997.

INDEX TERMS

Keywords:
3D reprojection, fundamental matrix, individually-focused teleconferencing (IFT), mirror reference view/image, three-view epipolar geometry