Continuous capture of skin deformation

ABSTRACT

We describe a method for the acquisition of deformable human geometry from silhouettes. Our technique uses a commercial tracking system to determine the motion of the skeleton, then estimates geometry for each bone using constraints provided by the silhouettes from one or more cameras. These silhouettes do not give a complete characterization of the geometry for a particular point in time, but when the subject moves, many observations of the same local geometries allow the construction of a complete model. Our reconstruction algorithm provides a simple mechanism for solving the problems of view aggregation, occlusion handling, hole filling, noise removal, and deformation modeling. The resulting model is parameterized to synthesize geometry for new poses of the skeleton. We demonstrate this capability by rendering the geometry for motion sequences that were not included in the original datasets.

REFERENCES

• 1: Brett Allen , Brian Curless , Zoran Popović, Articulated body deformation from range scan data, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 2: Volker Blanz , Thomas Vetter, A morphable model for the synthesis of 3D faces, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.187-194, July 1999 [doi>10.1145/311535.311556]
• 3: BRAND, M. 2001. Morphable 3D models from video. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), II:456--463.
• 4: BREGLER, C., HERTZMANN, A., AND BIERMANN, H. 2000. Recovering non-rigid 3D shape from image streams. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), II:690--696.
• 5: BROOMHEAD, D., AND LOWE, D. 1988. Multivariable functional interpolation and adaptive networks. Complex Systems 2, 3, 321--355.
• 6: J. C. Carr , R. K. Beatson , J. B. Cherrie , T. J. Mitchell , W. R. Fright , B. C. McCallum , T. R. Evans, Reconstruction and representation of 3D objects with radial basis functions, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.67-76, August 2001 [doi>10.1145/383259.383266]
• 7: Timothy F. Cootes , Gareth J. Edwards , Christopher J. Taylor, Active Appearance Models, Proceedings of the 5th European Conference on Computer Vision-Volume II, p.484-498, June 02-06, 1998
• 8: GU, X., GORTLER, S. J., HOPPE, H., MCMILLAN, L., BROWN, B. J., AND STONE, A. D. 1999. Silhouette mapping. Tech. Rep. TR-1-99, Harvard.
• 9: Brian Guenter , Cindy Grimm , Daniel Wood , Henrique Malvar , Fredric Pighin, Making faces, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.55-66, July 1998 [doi>10.1145/280814.280822]
• 10: I. A. Kakadiaris , D. Metaxas, 3D human body model acquisition from multiple views, Proceedings of the Fifth International Conference on Computer Vision, p.618, June 20-23, 1995
• 11: KAYDARA. 2001. FiLMBOX Reference Guide. Kaydara Inc., Montréal, Québec.
• 12: J. P. Lewis , Matt Cordner , Nickson Fong, Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.165-172, July 2000 [doi>10.1145/344779.344862]
• 13: Wojciech Matusik , Chris Buehler , Ramesh Raskar , Steven J. Gortler , Leonard McMillan, Image-based visual hulls, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.369-374, July 2000 [doi>10.1145/344779.344951]
• 14: Ivana Mikić , Mohan Trivedi , Edward Hunter , Pamela Cosman, Human Body Model Acquisition and Tracking Using Voxel Data, International Journal of Computer Vision, v.53 n.3, p.199-223, July-August 2003 [doi>10.1023/A:1023012723347]
• 15: NEBEL, J.-C., RODRIGUEZ-MIGUEL, F. J., AND COCKSHOTT, W. P. 2001. Stroboscopic stereo rangefinder. In Proceedings of the Third International Conference on 3D Imaging and Modeling, 59--64.
• 16: NELDER, J. A., AND MEAD, R. 1965. A simplex method for function minimization. Computer Journal 7, 4, 308--313.
• 17: NEVATIA, R., AND BINFORD, T. O. 1977. Description and recognition of curved objects. Artificial Intelligence 8, 1, 77--98.
• 18: PLÄNKERS, R., AND FUA, P. 2001. Articulated soft objects for video-based body modeling. In Proceedings of the 8th IEEE International Conference on Computer Vision (ICCV), I:394--401.
• 19: Szymon Rusinkiewicz , Olaf Hall-Holt , Marc Levoy, Real-time 3D model acquisition, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 20: Peter-Pike J. Sloan , Charles F. Rose, III , Michael F. Cohen, Shape by example, Proceedings of the 2001 symposium on Interactive 3D graphics, p.135-143, March 2001 [doi>10.1145/364338.364382]
• 21: STOKDYK, S., HAHN, K., NOFZ, P., AND ANDERSON, G., 2002. Spiderman: Behind the mask. Special Session of SIGGRAPH 2002.
• 22: Richard Szeliski , David Tonnesen, Surface modeling with oriented particle systems, Proceedings of the 19th annual conference on Computer graphics and interactive techniques, p.185-194, July 1992
• 23: Christian Theobalt , Marcus Magnor , Pascal Schüler , Hans-Peter Seidel, Combining 2D Feature Tracking and Volume Reconstruction for Online Video-Based Human Motion Capture, Proceedings of the 10th Pacific Conference on Computer Graphics and Applications, p.96, October 09-11, 2002
• 24: L. Torresani , Chris Bregler, Space-Time Tracking, Proceedings of the 7th European Conference on Computer Vision-Part I, p.801-812, May 28-31, 2002
• 25: Sundar Vedula , Simon Baker , Takeo Kanade, Spatio-temporal view interpolation, Proceedings of the 13th Eurographics workshop on Rendering, June 26-28, 2002, Pisa, Italy
• 26: VICON. 2003. Vicon iQ Reference Manual. Vicon Motion Systems Inc., Lake Forest, CA.
• 27: Xiaohuan Corina Wang , Cary Phillips, Multi-weight enveloping: least-squares approximation techniques for skin animation, Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 21-22, 2002, San Antonio, Texas [doi>10.1145/545261.545283]
• 28: Stephan Würmlin , Edouard Lamboray , Oliver G. Staadt , Markus H. Gross, 3D Video Recorder, Proceedings of the 10th Pacific Conference on Computer Graphics and Applications, p.325, October 09-11, 2002

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Animation

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Shape

Keywords:
human animation, motion capture, skin modeling