Marker-free kinematic skeleton estimation from sequences of volume data

ABSTRACT

For realistic animation of an artificial character a body model that represents the character's kinematic structure is required. Hierarchical skeleton models are widely used which represent bodies as chains of bones with interconnecting joints. In video motion capture, animation parameters are derived from the performance of a subject in the real world. For this acquisition procedure too, a kinematic body model is required. Typically, the generation of such a model for tracking and animation is, at best, a semi-automatic process. We present a novel approach that estimates a hierarchical skeleton model of an arbitrary moving subject from sequences of voxel data that were reconstructed from multi-view video footage. Our method does not require a-priori information about the body structure. We demonstrate its performance using synthetic and real data.

REFERENCES

• 1: Frederic Banégas , Marc Jaeger , Dominique Michelucci , M. Roelens, The ellipsoidal skeleton in medical applications, Proceedings of the sixth ACM symposium on Solid modeling and applications, p.30-38, May 2001, Ann Arbor, Michigan, United States [doi>10.1145/376957.376961]
• 2: Andrea Bottino , Aldo Laurentini, A silhoutte based technique for the reconstruction of human movement, Computer Vision and Image Understanding, v.83 n.1, p.79-95, July 2001 [doi>10.1006/cviu.2001.0918]
• 3: C. Bregler , J. Malik, Tracking People with Twists and Exponential Maps, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.8, June 23-25, 1998
• 4: Richard H. Byrd , Peihuang Lu , Jorge Nocedal , Ciyou Zhu, A limited memory algorithm for bound constrained optimization, SIAM Journal on Scientific Computing, v.16 n.5, p.1190-1208, Sept. 1995 [doi>10.1137/0916069]
• 5: J. Carranza, C. Theobalt, M. Magnor, and H.-P. Seidel. Free-viewpoint video of human actors. In Proc. of SIGGRAPH'03, pages 569--577, 2003.
• 6: G. Cheung, B. S., and T. Kanada. Shape-from-silhouette of articulated objects and its use for human body kinematics estiamtion and motion capture. In Proc. of CVPR, 2003.
• 7: K. Cheung, T. Kanade, J.-Y. Bouguet, and M. Holler. A real time system for robust 3D voxel reconstruction of human motions. In Proc. of CVPR, volume 2, pages 714--720, 2000.
• 8: L. Chevalier, F. Jaillet, and B. A. Segmentation and superquadric modeling of 3D objects. In Proc. of WSCG 2003, 2003.
• 9: Edilson de Aguiar , Christian Theobalt , Marcus Magnor , Holger Theisel , Hans-Peter Seidel, M^3: Marker-Free Model Reconstruction and Motion Tracking from 3D Voxel Data, Proceedings of the Computer Graphics and Applications, 12th Pacific Conference on (PG'04), p.101-110, October 06-08, 2004
• 10: Quentin Delamarre , Olivier Faugeras, 3D Articulated Models and Multi-View Tracking with Silhouettes, Proceedings of the International Conference on Computer Vision-Volume 2, p.716, September 20-25, 1999
• 11: B. Deutscher, A. Blake, and I. Reid. Articulated body motion capture by annealed particle filtering. In Proc. of CVPR'00, 2000.
• 12: D. M. Gavrila, The visual analysis of human movement: a survey, Computer Vision and Image Understanding, v.73 n.1, p.82-98, Jan. 1999 [doi>10.1006/cviu.1998.0716]
• 13: D. M. Gavrila , L. S. Davis, 3-D model-based tracking of humans in action: a multi-view approach, Proceedings of the 1996 Conference on Computer Vision and Pattern Recognition (CVPR '96), p.73, June 18-20, 1996
• 14: I. A. Kakadiaris , D. Metaxas, 3D human body model acquisition from multiple views, Proceedings of the Fifth International Conference on Computer Vision, p.618, June 20-23, 1995
• 15: S. Katz and A. Tal. Hierarchical mesh decomposition using fuzzy clustering and cuts. In Proc. of SIGGRAPH'03, pages 954--961, 2003.
• 16: Kiriakos N. Kutulakos , Steven M. Seitz, A Theory of Shape by Space Carving, International Journal of Computer Vision, v.38 n.3, p.199-218, July-August 2000 [doi>10.1023/A:1008191222954]
• 17: Aleš Leonardis , Aleš Jaklič , Franc Solina, Superquadrics for Segmenting and Modeling Range Data, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.11, p.1289-1295, November 1997 [doi>10.1109/34.632988]
• 18: Maylor K. Leung , Yee-Hong Yang, First Sight: A Human Body Outline Labeling System, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.17 n.4, p.359-377, April 1995 [doi>10.1109/34.385981]
• 19: S. Loncaric. A survey of shape analysis techniques. Pattern Recognition, 31(8):983--1001, 1998.
• 20: J. Luck and D. Small. Real-time markerless motion tracking using linked kinematic chains. In Proc. of CVPRIP02, 2002.
• 21: Alberto Menache , Alberto Manache, Understanding Motion Capture for Computer Animation and Video Games, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1999
• 22: I. Mikić, M. Triverdi, E. Hunter, and P. Cosman. Articulated body posture estimation from multicamera voxel data. In Proc. of CVPR, 2001.
• 23: Ralf Plänkers , Pascal Fua, Tracking and modeling people in video sequences, Computer Vision and Image Understanding, v.81 n.3, p.285-302, March 2001 [doi>10.1006/cviu.2000.0891]
• 24: William H. Press , William T. Vetterling , Saul A. Teukolsky , Brian P. Flannery, Numerical Recipes in C++: the art of scientific computing, 2001
• 25: K. Rohr. Incremental recognition of pedestrians from image sequences. In Proc. of CVPR, pages 8--13, 1993.
• 26: Marius-Calin Silaghi , Ralf Plänkers , Ronan Boulic , Pascal Fua , Daniel Thalmann, Local and Global Skeleton Fitting Techniques for Optical Motion Capture, Proceedings of the International Workshop on Modelling and Motion Capture Techniques for Virtual Environments, p.26-40, January 1998
• 27: Mashe Sniedovich, Dynamic Programming, Marcel Dekker, Inc., New York, NY, 1991
• 28: C. Theobalt, M. Li, M. Magnor, and H.-P. Seidel. A flexible and versatile studio for synchronized multi-view video recording. In Proc. of Vision, Video and Graphics, pages 9--16, 2003.
• 29: Christian Theobalt , Marcus Magnor , Pascal Schüler , Hans-Peter Seidel, Combining 2D Feature Tracking and Volume Reconstruction for Online Video-Based Human Motion Capture, Proceedings of the 10th Pacific Conference on Computer Graphics and Applications, p.96, October 09-11, 2002
• 30: Christopher Richard Wren , Ali Azarbayejani , Trevor Darrell , Alex Paul Pentland, Pfinder: Real-Time Tracking of the Human Body, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.7, p.780-785, July 1997 [doi>10.1109/34.598236]
• 31: S. Yonemoto , D. Arita , R. Taniguchi, Real-time human motion analysis and IK-based human figure control, Proceedings of the Workshop on Human Motion (HUMO'00), p.149, December 07-08, 2000

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Virtual reality

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Video analysis; Motion
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Time-varying imagery; Motion; Tracking
  I.5 PATTERN RECOGNITION
      I.5.1 Models
          Subjects: Structural

General Terms:
Algorithms, Measurement

Keywords:
kinematic skeleton, learning, model reconstruction, motion capture, tracking, volume processing