A real-time interactive multi-view video system

ABSTRACT

With the rapid development of electronic and computing technology, multi-view video is attracting extensive interest recently due to its greatly enhanced viewing experience. In this paper, we present the system architecture for real-time capturing, processing, and interactive delivery of multi-view video. Unlike previous systems that mainly focus on multi-view video capturing, our system is designed to provide multi-view video service with high degree of interactivity in real time, which is still challenging in the current state of the technology. The proposed architecture tackles many practical problems in system calibration, object tracking, video compression, interactive delivery, etc. With the proposed system, users can interactively select their desired viewing directions and enjoy many exciting visual experiences, such as view switching, frozen moment and view sweeping, in real-time and with great freedom.

REFERENCES

• 1: Applications and requirements for 3dav. ISO/IEC JTC1/SC29/WG11, N5877, July 2003.
• 2: Report on 3dav exploration. ISO/IEC JTC1/SC29/WG11, N5878, July 2003.
• 3: Vasudev Bhaskaran , Konstantinos Konstantinides, Image and Video Compression Standards: Algorithms and Architectures, Kluwer Academic Publishers, Norwell, MA, 1997
• 4: R. Blahut. Theory and practice of error control codes. Addison-Wesley, 1993.
• 5: I. J. Cox , S. Roy , S. L. Hingorani, Dynamic histogram warping of image pairs for constant image brightness, Proceedings of the 1995 International Conference on Image Processing (Vol.2)-Volume 2, p.2366, October 23-26, 1995
• 6: P. Debevec, Y. Yu, and G. Borshukov. Efficient view-dependent image-based rendering with projective texture mapping. ACM SIGGRAPH.
• 7: Richard Hartley , Andrew Zisserman, Multiple view geometry in computer visiond, Cambridge University Press, New York, NY, 2001
• 8: R. Li, B. Zeng, and M. Liou. A new three-step search algorithm for block motion estimation. IEEE Transactions on Circuits and Systems for Video Technology, 4(4):438--442, Aug. 1994.
• 9: J.-R. Ohm and K. Muller. Incomplete 3d representation of video objects for multiview applications. PC97.
• 10: H. Tao, H. S. Sawhney, and R. Kumar. A global matching framework for stereo computation. In International Conferece of Computer Vision. IEEE, 2001.
• 11: R. Y. Tsai. An efficient and accurate camera calibration technique for 3d machine vision. In International Conferece of Computer Vision and Pattern Recognition, pages 364--374. IEEE, September 1986.
• 12: Stephan Würmlin , Edouard Lamboray , Oliver G. Staadt , Markus H. Gross, 3D Video Recorder, Proceedings of the 10th Pacific Conference on Computer Graphics and Applications, p.325, October 09-11, 2002
• 13: Z. Yang, H. Cai, and J. Li. A framework for fine-granular computational-complexity scalable motion estimation. accepted by ISCAS 2005, May 2005.
• 14: Zhengyou Zhang, A Flexible New Technique for Camera Calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.11, p.1330-1334, November 2000 [doi>10.1109/34.888718]
• 15: S. Zhu and K. Ma. A new diamond search algorithm for fast block-matching motion estimation. IEEE Transactions on Image Processing, 9(2):287--290, Feb. 2000.
• 16: C. L. Zitinick, S. Kang, M. Uyttendaele, S. Winder, and R. Szeliski. High-quality video view interpolation using a layered representation. In SIGGRAPH, pages 600--608. ACM, August 2004.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.9 Applications

General Terms:
Design, Performance, Standardization

Keywords:
calibration, channel coding, multi-view video, object tracking, source coding, video coding, video streaming system