Style translation for human motion

ABSTRACT

Style translation is the process of transforming an input motion into a new style while preserving its original content. This problem is motivated by the needs of interactive applications, which require rapid processing of captured performances. Our solution learns to translate by analyzing differences between performances of the same content in input and output styles. It relies on a novel correspondence algorithm to align motions, and a linear time-invariant model to represent stylistic differences. Once the model is estimated with system identification, our system is capable of translating streaming input with simple linear operations at each frame.

REFERENCES

• 1: Kenji Amaya , Armin Bruderlin , Tom Calvert, Emotion from motion, Proceedings of the conference on Graphics interface '96, p.222-229, May 1996, Toronto, Ontario, Canada
• 2: Okan Arikan , D. A. Forsyth, Interactive motion generation from examples, ACM Transactions on Graphics (TOG), v.21 n.3, July 2002
• 3: Okan Arikan , David A. Forsyth , James F. O'Brien, Motion synthesis from annotations, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 4: Matthew Brand , Aaron Hertzmann, Style machines, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.183-192, July 2000 [doi>10.1145/344779.344865]
• 5: Brockwell, P. J., and Davis, R. A. 2002. Introduction to Time Series and Forecasting, 2nd ed. Springer-Verlag.
• 6: Armin Bruderlin , Lance Williams, Motion signal processing, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.97-104, September 1995 [doi>10.1145/218380.218421]
• 7: De La Torre, F., and Black, M. 2001. Dynamic coupled component analysis. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 643--650.
• 8: Mira Dontcheva , Gary Yngve , Zoran Popović, Layered acting for character animation, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 9: Richard O. Duda , Peter E. Hart , David G. Stork, Pattern Classification (2nd Edition), Wiley-Interscience, 2000
• 10: Petros Faloutsos , Michiel van de Panne , Demetri Terzopoulos, Composable controllers for physics-based character animation, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.251-260, August 2001 [doi>10.1145/383259.383287]
• 11: William T. Freeman , Joshua B. Tenenbaum , Egon C. Pasztor, Learning style translation for the lines of a drawing, ACM Transactions on Graphics (TOG), v.22 n.1, p.33-46, January 2003 [doi>10.1145/588272.588277]
• 12: Martin A. Giese , Tomaso Poggio, Morphable Models for the Analysis and Synthesis of Complex Motion Patterns, International Journal of Computer Vision, v.38 n.1, p.59-73, June 2000 [doi>10.1023/A:1008118801668]
• 13: Michael Gleicher, Retargetting motion to new characters, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.33-42, July 1998 [doi>10.1145/280814.280820]
• 14: F. Sebastin Grassia, Practical parameterization of rotations using the exponential map, Journal of Graphics Tools, v.3 n.3, p.29-48, 1 March 1998
• 15: Keith Grochow , Steven L. Martin , Aaron Hertzmann , Zoran Popović, Style-based inverse kinematics, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004
• 16: Aaron Hertzmann , Charles E. Jacobs , Nuria Oliver , Brian Curless , David H. Salesin, Image analogies, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.327-340, August 2001 [doi>10.1145/383259.383295]
• 17: Eugene Hsu , Sommer Gentry , Jovan Popović, Example-based control of human motion, Proceedings of the 2004 ACM SIGGRAPH/Eurographics symposium on Computer animation, August 27-29, 2004, Grenoble, France [doi>10.1145/1028523.1028534]
• 18: Winfried Ilg , Martin A. Giese, Modeling of Movement Sequences Based on Hierarchical Spatial-Temporal Correspondence of Movement Primitives, Proceedings of the Second International Workshop on Biologically Motivated Computer Vision, p.528-537, November 22-24, 2002
• 19: Tony Jebara , Alex Pentland, Action Reaction Learning: Automatic Visual Analysis and Synthesis of Interactive Behaviour, Proceedings of the First International Conference on Computer Vision Systems, p.273-292, January 13-15, 1999
• 20: Lucas Kovar , Michael Gleicher, Flexible automatic motion blending with registration curves, Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 26-27, 2003, San Diego, California
• 21: Lucas Kovar , Michael Gleicher, Automated extraction and parameterization of motions in large data sets, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004
• 22: Lucas Kovar , Michael Gleicher , Frédéric Pighin, Motion graphs, ACM Transactions on Graphics (TOG), v.21 n.3, July 2002
• 23: Lucas Kovar , John Schreiner , Michael Gleicher, Footskate cleanup for motion capture editing, Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 21-22, 2002, San Antonio, Texas [doi>10.1145/545261.545277]
• 24: Jehee Lee , Sung Yong Shin, A hierarchical approach to interactive motion editing for human-like figures, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.39-48, July 1999 [doi>10.1145/311535.311539]
• 25: Jehee Lee , Jinxiang Chai , Paul S. A. Reitsma , Jessica K. Hodgins , Nancy S. Pollard, Interactive control of avatars animated with human motion data, ACM Transactions on Graphics (TOG), v.21 n.3, July 2002
• 26: Yan Li , Tianshu Wang , Heung-Yeung Shum, Motion texture: a two-level statistical model for character motion synthesis, ACM Transactions on Graphics (TOG), v.21 n.3, July 2002
• 27: Lennart Ljung, System identification (2nd ed.): theory for the user, Prentice Hall PTR, Upper Saddle River, NJ, 1999
• 28: Luenberger, D. G. 1979. Introduction to Dynamic Systems: Theory, Models, and Applications, 1st ed. Wiley.
• 29: Sang Il Park , Hyun Joon Shin , Sung Yong Shin, On-line locomotion generation based on motion blending, Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 21-22, 2002, San Antonio, Texas [doi>10.1145/545261.545279]
• 30: Sang Il Park , Hyun Joon Shin , Tae Hoon Kim , Sung Yong Shin, On-line motion blending for real-time locomotion generation: Research Articles, Computer Animation and Virtual Worlds, v.15 n.3-4, p.125-138, July 2004 [doi>10.1002/cav.v15:3/4]
• 31: Ken Perlin, Real Time Responsive Animation with Personality, IEEE Transactions on Visualization and Computer Graphics, v.1 n.1, p.5-15, March 1995 [doi>10.1109/2945.468392]
• 32: Zoran Popović , Andrew Witkin, Physically based motion transformation, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.11-20, July 1999 [doi>10.1145/311535.311536]
• 33: Katherine Pullen , Christoph Bregler, Motion capture assisted animation: texturing and synthesis, ACM Transactions on Graphics (TOG), v.21 n.3, July 2002
• 34: Lawrence Rabiner , Biing-Hwang Juang, Fundamentals of speech recognition, Prentice-Hall, Inc., Upper Saddle River, NJ, 1993
• 35: Charles Rose , Michael F. Cohen , Bobby Bodenheimer, Verbs and Adverbs: Multidimensional Motion Interpolation, IEEE Computer Graphics and Applications, v.18 n.5, p.32-40, September 1998 [doi>10.1109/38.708559]
• 36: Hyun Joon Shin , Jehee Lee , Sung Yong Shin , Michael Gleicher, Computer puppetry: An importance-based approach, ACM Transactions on Graphics (TOG), v.20 n.2, p.67-94, April 2001 [doi>10.1145/502122.502123]
• 37: Soatto, S., Doretto, G., and Wu, Y. N. 2001. Dynamic textures. In International Conference on Computer Vision (ICCV), 439--446.
• 38: Stengel, R. F. 1994. Optimal Control and Estimation. Dover Books on Advanced Mathematics, New York, NY.
• 39: David J. Sturman, Computer Puppetry, IEEE Computer Graphics and Applications, v.18 n.1, p.38-45, January 1998 [doi>10.1109/38.637269]
• 40: Joshua B. Tenenbaum , William T. Freeman, Separating Style and Content with Bilinear Models, Neural Computation, v.12 n.6, p.1247-1283, June 2000 [doi>10.1162/089976600300015349]
• 41: Munetoshi Unuma , Ken Anjyo , Ryozo Takeuchi, Fourier principles for emotion-based human figure animation, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.91-96, September 1995 [doi>10.1145/218380.218419]
• 42: Van Overschee, P., and De Moor, B. 1996. Subspace Identification for Linear Systems: Theory, Implementation, Applications. Kluwer Academic Publishers, Dordrecht, Netherlands.
• 43: M. Alex O. Vasilescu, Human Motion Signatures: Analysis, Synthesis, Recognition, Proceedings of the 16 th International Conference on Pattern Recognition (ICPR'02) Volume 3, p.30456, August 11-15, 2002
• 44: Andrew Witkin , Zoran Popovic, Motion warping, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.105-108, September 1995 [doi>10.1145/218380.218422]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Animation

Keywords:
data mining, human simulation, machine learning