Style machines

ABSTRACT

We approach the problem of stylistic motion synthesis by learning motion patterns from a highly varied set of motion capture sequences. Each sequence may have a distinct choreography, performed in a distinct sytle. Learning identifies common choreographic elements across sequences, the different styles in which each element is performed, and a small number of stylistic degrees of freedom which span the many variations in the dataset. The learned model can synthesize novel motion data in any interpolation or extrapolation of styles. For example, it can convert novice ballet motions into the more graceful modern dance of an expert. The model can also be driven by video, by scripts or even by noise to generate new choreography and synthesize virtual motion-capture in many styles.

REFERENCES

• 1: L. Baum. An inequality and associated maximization technique in statistical estimation of probabilistic functions of Markov processes. Inequalities, 3:1-8, 1972.
• 2: M. Brand. Pattern discovery via entropy minimization. In D. Heckerman and C. Whittaker, editors, Artificial Intelligence and Statistics #7. Morgan Kaufmann., January 1999.
• 3: Matthew Brand, Finding Variational Structure in Data by Cross-Entropy Optimization, Proceedings of the Seventeenth International Conference on Machine Learning, p.95-102, June 29-July 02, 2000
• 4: Matthew Brand, Voice puppetry, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.21-28, July 1999 [doi>10.1145/311535.311537]
• 5: Matthew Brand, Shadow Puppetry, Proceedings of the International Conference on Computer Vision-Volume 2, p.1237, September 20-25, 1999
• 6: Christoph Bregler, Learning and Recognizing Human Dynamics in Video Sequences, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.568, June 17-19, 1997
• 7: Armin Bruderlin , Lance Williams, Motion signal processing, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.97-104, September 1995 [doi>10.1145/218380.218421]
• 8: Joachim M. Buhmann, Empirical Risk Approximation: An Induction Principle for Unsupervised Learning, University of Bonn, 1998
• 9: R.M. Corless, G. H. Gonnet, D. E. G. Hare, D. J. Jeffrey, and D. E. Knuth. On the Lambert W function. Advances in Computational Mathematics, 5:329-359, 1996.
• 10: William T. Freeman , Joshua B. Tenenbaum, Learning bilinear models for two-factor problems in vision., Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.554, June 17-19, 1997
• 11: Michael Gleicher, Motion editing with spacetime constraints, Proceedings of the 1997 symposium on Interactive 3D graphics, p.139-ff., April 27-30, 1997, Providence, Rhode Island, United States [doi>10.1145/253284.253321]
• 12: Michael Gleicher, Retargetting motion to new characters, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.33-42, July 1998 [doi>10.1145/280814.280820]
• 13: Radek Grzeszczuk , Demetri Terzopoulos , Geoffrey Hinton, NeuroAnimator: fast neural network emulation and control of physics-based models, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.9-20, July 1998 [doi>10.1145/280814.280816]
• 14: N.R. Howe, M. E. Leventon, and W. T. Freeman. Bayesian reconstruction of 3d human motion from single-camera video. In S. Solla, T. Leend, and K. Muller, editors, Advances in Neural Information P~vcessing Systems, volume 10. MIT Press, 2000.
• 15: Jehee Lee , Sung Yong Shin, A hierarchical approach to interactive motion editing for human-like figures, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.39-48, July 1999 [doi>10.1145/311535.311539]
• 16: Zoran Popović , Andrew Witkin, Physically based motion transformation, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.11-20, July 1999 [doi>10.1145/311535.311536]
• 17: L.R. Rabiner. A tutorial on hidden Markov models and selected applications in speech recognition. P1vceedings of the IEEE, 77(2):257-286, Feb. 1989.
• 18: Charles Rose , Michael F. Cohen , Bobby Bodenheimer, Verbs and Adverbs: Multidimensional Motion Interpolation, IEEE Computer Graphics and Applications, v.18 n.5, p.32-40, September 1998 [doi>10.1109/38.708559]
• 19: J.B. Tenenbaum and W. T. Freeman. Separating style and content. In M. Mozer, M. Jordan, and T. Petsche, editors, Advances in Neural Information P~vcessing Systems, volume 9, pages 662-668. MIT Press, 1997.
• 20: Munetoshi Unuma , Ken Anjyo , Ryozo Takeuchi, Fourier principles for emotion-based human figure animation, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.91-96, September 1995 [doi>10.1145/218380.218419]
• 21: Andrew D. Wilson , Aaron F. Bobick, Parametric Hidden Markov Models for Gesture Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.21 n.9, p.884-900, September 1999 [doi>10.1109/34.790429]
• 22: Andrew Witkin , Zoran Popovic, Motion warping, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.105-108, September 1995 [doi>10.1145/218380.218422]
• 23: Song Chun Zhu , Ying Nian Wu , David Mumford, Minimax entropy principle and its application to texture modeling, Neural Computation, v.9 n.9, p.1627-1660, Nov. 15, 1997 [doi>10.1162/neco.1997.9.8.1627]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Animation

Additional Classification:
  E. Data
  E.4 CODING AND INFORMATION THEORY
      Subjects: Data compaction and compression

  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Time series analysis

  H. Information Systems
  H.1 MODELS AND PRINCIPLES

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Motion
      I.2.9 Robotics
          Subjects: Kinematics and dynamics
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Motion

  J. Computer Applications
  J.5 ARTS AND HUMANITIES
      Subjects: Performing arts (e.g., dance, music)

General Terms:
Algorithms, Design, Human Factors, Measurement, Performance, Theory

Keywords:
animation, behavior simulation, character behavior