Inserting 3D projected virtual content into broadcast tennis video

ABSTRACT

The ability to acquire the accurate camera matrix of each frame of a video clip is essential if virtual content is inserted into the images in a believable way. This paper presents our system for inserting projected virtual content into broadcast tennis video based on camera matrix acquired. To achieve the accurate camera matrix, we develop a new algorithm for the 3D camera calibration of broadcast tennis video, which improves the accuracy of camera matrices via the proposed techniques of clip-wise data analysis and Hough-like search. We divide all the camera parameters determining a camera matrix into two categories: clip-varying and frame-varying. For the clip-varying ones, we use a clip-wise data analysis procedure to achieve their accuracy. For the framevarying ones, we use a Hough-like search to tune them for each frame. Preliminary experiments results show that we can seamlessly insert projected virtual content into each frame.

REFERENCES

• 1: D. Farin, S. Krabbe, P. H. N. de With, and W. Effelsberg. Robust camera calibration for sport videos using court models. SPIE Storage & Retr. M. & App. for MM, vol 5307, pp80--91, 2004.
• 2: Simon Gibson , Jon Cook , Toby Howard , Roger Hubbold , Dan Oram, Accurate Camera Calibration for Off-line, Video-Based Augmented Reality, Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR'02), p.37, September 30-October 01, 2002
• 3: J. R. Gloundemans, et al. Blending a graphic, US paten: 6,229,550 (From Sportsvision Inc, US).
• 4: Richard Hartley , Andrew Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, New York, NY, 2003
• 5: Ian D. Reid , Andrew Zisserman, Goal-directed Video Metrology, Proceedings of the 4th European Conference on Computer Vision-Volume II, p.647-658, April 15-18, 1996
• 6: R. Y. Tsai. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the shelf TV cameras and lenses, IEEE J. of Robotics and Automation, vol. RA-3, No. 4, Aug 1987, pp 323--344.
• 7: Kongwah WAN , Xin YAN , Xinguo YU , Changsheng XU, Robust goal-mouth detection for virtual content insertion, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957116]
• 8: X. Yu, C. H. Sim, J. R. Wang, and F. C. Loong. A trajectory-based ball detection and tracking algorithm in broadcast tennis video. ICIP04, pp1049--1052, 2004.
• 9: http://www.orad.co.il
• 10: http://www.sportvision.com
• 11: http://www.hawkeyeinnovations.co.uk.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Object recognition

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Shape

General Terms:
Algorithms, Experimentation, Measurement

Keywords:
camera calibration, data analysis, hough-like search, sports video, virtual content insertion