Human video textures

ABSTRACT

This paper describes a data-driven approach for generating photorealistic animations of human motion. Each animation sequence follows a user-choreographed path and plays continuously by seamlessly transitioning between different segments of the captured data. To produce these animations, we capitalize on the complementary characteristics of motion capture data and video. We customize our capture system to record motion capture data that are synchronized with our video source. Candidate transition points in video clips are identified using a new similarity metric based on 3-D marker trajectories and their 2-D projections into video. Once the transitions have been identified, a video-based motion graph is constructed. We further exploit hybrid motion and video data to ensure that the transitions are seamless when generating animations. Motion capture marker projections serve as control points for segmentation of layers and nonrigid transformation of regions. This allows warping and blending to generate seamless in-between frames for animation. We show a series of choreographed animations of walks and martial arts scenes as validation of our approach.

REFERENCES

• 1: Aseem Agarwala , Ke Colin Zheng , Chris Pal , Maneesh Agrawala , Michael Cohen , Brian Curless , David Salesin , Richard Szeliski, Panoramic video textures, ACM Transactions on Graphics (TOG), v.24 n.3, July 2005
• 2: Thaddeus Beier , Shawn Neely, Feature-based image metamorphosis, Proceedings of the 19th annual conference on Computer graphics and interactive techniques, p.35-42, July 1992
• 3: Paul J. Besl , Neil D. McKay, A Method for Registration of 3-D Shapes, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.14 n.2, p.239-256, February 1992 [doi>10.1109/34.121791]
• 4: Borshukov, G., Hable, J., and Montgomery, J. 2007. Playable Universal Capture chapter in GPU Gems 3. Addison Wesley. 6
• 5: Yuri Boykov , Vladimir Kolmogorov, An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.26 n.9, p.1124-1137, September 2004 [doi>10.1109/TPAMI.2004.60]
• 6: Joel Carranza , Christian Theobalt , Marcus A. Magnor , Hans-Peter Seidel, Free-viewpoint video of human actors, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 7: Celly, B., and Zordan, V. 2004. Animated people textures. In 17th International Conference on Computer Animation and Social Agents. 2, 3
• 8: Cobzas, D., Yerex, K., and Jgersand, M. 2002. Dynamic textures for image-based rendering of fine-scale 3d structure and animation of non-rigid motion. In Eurographics, 1067--7055. 3
• 9: Edilson de Aguiar , Carsten Stoll , Christian Theobalt , Naveed Ahmed , Hans-Peter Seidel , Sebastian Thrun, Performance capture from sparse multi-view video, ACM Transactions on Graphics (TOG), v.27 n.3, August 2008
• 10: Alexei A. Efros , Thomas K. Leung, Texture Synthesis by Non-Parametric Sampling, Proceedings of the International Conference on Computer Vision-Volume 2, p.1033, September 20-25, 1999
• 11: Pedro F. Felzenszwalb , Daniel P. Huttenlocher, Efficient Graph-Based Image Segmentation, International Journal of Computer Vision, v.59 n.2, p.167-181, September 2004 [doi>10.1023/B:VISI.0000022288.19776.77]
• 12: Richard Hartley , Andrew Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, New York, NY, 2003
• 13: Alexander Hornung , Ellen Dekkers , Leif Kobbelt, Character animation from 2D pictures and 3D motion data, ACM Transactions on Graphics (TOG), v.26 n.1, p.1-es, January 2007 [doi>10.1145/1189762.1189763]
• 14: Lucas Kovar , Michael Gleicher , Frédéric Pighin, Motion graphs, ACM Transactions on Graphics (TOG), v.21 n.3, July 2002
• 15: S. Z. Li, Markov random field modeling in computer vision, Springer-Verlag, London, 1995
• 16: Peter Sand , Leonard McMillan , Jovan Popović, Continuous capture of skin deformation, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 17: Scott Schaefer , Travis McPhail , Joe Warren, Image deformation using moving least squares, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
• 18: Arno Schödl , Irfan A. Essa, Controlled animation of video sprites, Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 21-22, 2002, San Antonio, Texas [doi>10.1145/545261.545281]
• 19: Arno Schödl , Richard Szeliski , David H. Salesin , Irfan Essa, Video textures, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.489-498, July 2000 [doi>10.1145/344779.345012]
• 20: Jonathan Starck , Adrian Hilton, Surface Capture for Performance-Based Animation, IEEE Computer Graphics and Applications, v.27 n.3, p.21-31, May 2007 [doi>10.1109/MCG.2007.68]
• 21: Zhao, L., and Safonova, A. 2008. Achieving good connectivity in motion graphs. In Proceedings of the 2008 ACM/Eurographics Symposium on Computer Animation. 7
• 22: C. Lawrence Zitnick , Sing Bing Kang , Matthew Uyttendaele , Simon Winder , Richard Szeliski, High-quality video view interpolation using a layered representation, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Animation

Keywords:
image-based rendering, layered motion, motion capture