Precomputing avatar behavior from human motion data

ABSTRACT

Creating controllable, responsive avatars is an important problem in computer games and virtual environments. Recently, large collections of motion capture data have been exploited for increased realism in avatar animation and control. Large motion sets have the advantage of accommodating a broad variety of natural human motion. However, when a motion set is large, the time required to identify an appropriate sequence of motions is the bottleneck for achieving interactive avatar control. In this paper, we present a novel method of precomputing avatar behavior from unlabelled motion data in order to animate and control avatars at minimal runtime cost. Based on dynamic programming, our method finds a control policy that indicates how the avatar should act in any given situation. We demonstrate the effectiveness of our approach through examples that include avatars interacting with each other and with the user.

REFERENCES

• 1: Okan Arikan , D. A. Forsyth, Interactive motion generation from examples, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 2: Okan Arikan , David A. Forsyth , James F. O'Brien, Motion synthesis from annotations, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 3: Christopher G. Atkeson , Andrew W. Moore , Stefan Schaal, Locally Weighted Learning for Control, Artificial Intelligence Review, v.11 n.1-5, p.75-113, Feb. 1997 [doi>10.1023/A:1006511328852]
• 4: A. Bruderlin , T. W. Calvert, Goal-directed, dynamic animation of human walking, Proceedings of the 16th annual conference on Computer graphics and interactive techniques, p.233-242, July 1989
• 5: Bruce Blumberg , Marc Downie , Yuri Ivanov , Matt Berlin , Michael Patrick Johnson , Bill Tomlinson, Integrated learning for interactive synthetic characters, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 6: Bruce M. Blumberg , Tinsley A. Galyean, Multi-level direction of autonomous creatures for real-time virtual environments, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.47-54, September 1995 [doi>10.1145/218380.218405]
• 7: Matthew Brand , Aaron Hertzmann, Style machines, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.183-192, July 2000 [doi>10.1145/344779.344865]
• 8: {BHG93} Badler N. I., Hollick M., Granieri J.: Real-time control of a virtual human using minimal sensors. Presence 2 (1993), 82--86.
• 9: Bruce M. Blumberg, Swamped! using plush toys to direct autonomous animated characters, ACM SIGGRAPH 98 Conference abstracts and applications, p.109, July 19-24, 1998, Orlando, Florida, United States [doi>10.1145/280953.281021]
• 10: {Bow00} Bowden R.: Learning statistical models of human motion. In IEEE Workshop on Human Modelling, Analysis and Synthesis, CVPR2000 (July 2000).
• 11: {BS97} Bradley E., Stuart J.: Using chaos to generate choreographic variations. In Proceedings of the Experimental Chaos Conference (august 1997).
• 12: Min Gyu Choi , Jehee Lee , Sung Yong Shin, Planning biped locomotion using motion capture data and probabilistic roadmaps, ACM Transactions on Graphics (TOG), v.22 n.2, p.182-203, April 2003 [doi>10.1145/636886.636889]
• 13: Mira Dontcheva , Gary Yngve , Zoran Popović, Layered acting for character animation, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 14: Petros Faloutsos , Michiel van de Panne , Demetri Terzopoulos, Composable controllers for physics-based character animation, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.251-260, August 2001 [doi>10.1145/383259.383287]
• 15: Aphrodite Galata , Neil Johnson , David Hogg, Learning variable-length Markov models of behavior, Computer Vision and Image Understanding, v.81 n.3, p.398-413, March 2001 [doi>10.1006/cviu.2000.0894]
• 16: Radek Grzeszczuk , Demetri Terzopoulos, Automated learning of muscle-actuated locomotion through control abstraction, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.63-70, September 1995 [doi>10.1145/218380.218411]
• 17: Radek Grzeszczuk , Demetri Terzopoulos , Geoffrey Hinton, NeuroAnimator: fast neural network emulation and control of physics-based models, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.9-20, July 1998 [doi>10.1145/280814.280816]
• 18: Doug L. James , Kayvon Fatahalian, Precomputing interactive dynamic deformable scenes, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 19: Lucas Kovar , Michael Gleicher , Frédéric Pighin, Motion graphs, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 20: {KLM96} Kaelbling L. P., Littman M. L., Moore A. W.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4 (1996), 237--285.
• 21: Tae-hoon Kim , Sang Il Park , Sung Yong Shin, Rhythmic-motion synthesis based on motion-beat analysis, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 22: Jehee Lee , Jinxiang Chai , Paul S. A. Reitsma , Jessica K. Hodgins , Nancy S. Pollard, Interactive control of avatars animated with human motion data, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 23: Yan Li , Tianshu Wang , Heung-Yeung Shum, Motion texture: a two-level statistical model for character motion synthesis, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 24: Andrew W. Moore , Christopher G. Atkeson, The Parti-game Algorithm for Variable Resolution Reinforcement Learning in Multidimensional State-spaces, Machine Learning, v.21 n.3, p.199-233, Dec. 1995 [doi>10.1023/A:1022656217772]
• 25: {Mat94} Mataric M. J.: Reward functions for accelerated learning. In Proceedings of the Eleventh International Conference on Machine Learning (1994).
• 26: Tom Molet , Ronan Boulic , Daniel Thalmann, A real time anatomical converter for human motion capture, Proceedings of the Eurographics workshop on Computer animation and simulation '96, p.79-94, December 1996, Poitiers, France
• 27: L. M. Tanco , A. Hilton, Realistic synthesis of novel human movements from a database of motion capture examples, Proceedings of the Workshop on Human Motion (HUMO'00), p.137, December 07-08, 2000
• 28: J. Thomas Ngo , Joe Marks, Spacetime constraints revisited, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.343-350, September 1993 [doi>10.1145/166117.166160]
• 29: Ren Ng , Ravi Ramamoorthi , Pat Hanrahan, All-frequency shadows using non-linear wavelet lighting approximation, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 30: Tsukasa Noma , Liwei Zhao , Norman I. Badler, Design of a Virtual Human Presenter, IEEE Computer Graphics and Applications, v.20 n.4, p.79-85, July 2000 [doi>10.1109/38.851755]
• 31: Katherine Pullen , Christoph Bregler, Animating by Multi-Level Sampling, Proceedings of the Computer Animation, p.36, May 03-05, 2000
• 32: Katherine Pullen , Christoph Bregler, Motion capture assisted animation: texturing and synthesis, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
• 33: Ken Perlin , Athomas Goldberg, Improv: a system for scripting interactive actors in virtual worlds, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.205-216, August 1996 [doi>10.1145/237170.237258]
• 34: Richard S. Sutton , Andrew G. Barto, Introduction to Reinforcement Learning, MIT Press, Cambridge, MA, 1998
• 35: Hedvig Sidenbladh , Michael J. Black , Leonid Sigal, Implicit Probabilistic Models of Human Motion for Synthesis and Tracking, Proceedings of the 7th European Conference on Computer Vision-Part I, p.784-800, May 28-31, 2002
• 36: {SE01} Schödl A., Essa I.: Machine learning for video-based rendering. In Advances in Neural Information Processing Systems (NIPS) (2001), vol. 13, pp. 1002--1008.
• 37: Arno Schödl , Irfan A. Essa, Controlled animation of video sprites, Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 21-22, 2002, San Antonio, Texas [doi>10.1145/545261.545281]
• 38: Peter-Pike Sloan , Jesse Hall , John Hart , John Snyder, Clustered principal components for precomputed radiance transfer, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 39: {SHS98} Semwal S., Hightower R., Stansfield S.: Mapping algorithms for real-time control of an avatar using eight sensors. Presence 7, 1 (1998), 1--21.
• 40: Karl Sims, Evolving virtual creatures, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.15-22, July 1994 [doi>10.1145/192161.192167]
• 41: Hyun Joon Shin , Jehee Lee , Sung Yong Shin , Michael Gleicher, Computer puppetry: An importance-based approach, ACM Transactions on Graphics (TOG), v.20 n.2, p.67-94, April 2001 [doi>10.1145/502122.502123]
• 42: Peter-Pike Sloan , Xinguo Liu , Heung-Yeung Shum , John Snyder, Bi-scale radiance transfer, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 43: Arno Schödl , Richard Szeliski , David H. Salesin , Irfan Essa, Video textures, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.489-498, July 2000 [doi>10.1145/344779.345012]
• 44: Victor Brian Zordan , Jessica K. Hodgins, Motion capture-driven simulations that hit and react, Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 21-22, 2002, San Antonio, Texas [doi>10.1145/545261.545276]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism