Animal gaits from video

ABSTRACT

We present a method for animating 3D models of animals from existing live video sequences such as wild life documentaries. Videos are first segmented into binary images on which Principal Component Analysis (PCA) is applied. The time-varying coordinates of the images in the PCA space are then used to generate 3D animation. This is done through interpolation with Radial Basis Functions (RBF) of 3D pose examples associated with a small set of key-images extracted from the video. In addition to this processing pipeline, our main contributions are: an automatic method for selecting the best set of key-images for which the designer will need to provide 3D pose examples. This method saves user time and effort since there is no more need for manual selection within the video and then trials and errors in the choice of key-images and 3D pose examples. As another contribution, we propose a simple algorithm based on PCA images to resolve 3D pose prediction ambiguities. These ambiguities are inherent to many animal gaits when only monocular view is available.

The method is first evaluated on sequences of synthetic images of animal gaits, for which full 3D data is available. We achieve a good quality reconstruction of the input 3D motion from a single video sequence of its 2D rendering. We then illustrate the method by reconstructing animal gaits from live video of wild life documentaries.

REFERENCES

• 1: {AM00} Alexa M., Müller W.: Representing animation by principal components. In Proc. EURO-GRAPHICS'00 (2000).
• 2: Christoph Bregler , Lorie Loeb , Erika Chuang , Hrishi Deshpande, Turning to the masters: motion capturing cartoons, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 3: James Davis , Maneesh Agrawala , Erika Chuang , Zoran Popović , David Salesin, A sketching interface for articulated figure animation, Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 26-27, 2003, San Diego, California
• 4: Michael Gleicher , Nicola Ferrier, Evaluating Video-Based Motion Capture, Proceedings of the Computer Animation, p.75, June 19-21, 2002
• 5: J. P. Lewis , Matt Cordner , Nickson Fong, Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.165-172, July 2000 [doi>10.1145/344779.344862]
• 6: Hyewon Pyun , Yejin Kim , Wonseok Chae , Hyung Woo Kang , Sung Yong Shin, An example-based approach for facial expression cloning, Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 26-27, 2003, San Diego, California
• 7: Charles Rose , Michael F. Cohen , Bobby Bodenheimer, Verbs and Adverbs: Multidimensional Motion Interpolation, IEEE Computer Graphics and Applications, v.18 n.5, p.32-40, September 1998 [doi>10.1109/38.708559]
• 8: Peter-Pike J. Sloan , Charles F. Rose, III , Michael F. Cohen, Shape by example, Proceedings of the 2001 symposium on Interactive 3D graphics, p.135-143, March 2001 [doi>10.1145/364338.364382]
• 9: Jianbo Shi , Jitendra Malik, Normalized Cuts and Image Segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.8, p.888-905, August 2000 [doi>10.1109/34.868688]
• 10: {TP91} Turk M., Pentland A.: Eigen faces for recognition. Journal of Cognitive Neuroscience 3, 1 (1991).
• 11: {WG03} Wilhelms J., Gelder A. V.: Combining vision and computer graphics for video motion capture. The Visual Computer 19, 6 (Oct 2003).

INDEX TERMS

Keywords:
animation from motion/video data, interpolation keyframing, intuitive interfaces for animation