Interactive control of avatars animated with human motion data

ABSTRACT

Real-time control of three-dimensional avatars is an important problem in the context of computer games and virtual environments. Avatar animation and control is difficult, however, because a large repertoire of avatar behaviors must be made available, and the user must be able to select from this set of behaviors, possibly with a low-dimensional input device. One appealing approach to obtaining a rich set of avatar behaviors is to collect an extended, unlabeled sequence of motion data appropriate to the application. In this paper, we show that such a motion database can be preprocessed for flexibility in behavior and efficient search and exploited for real-time avatar control. Flexibility is created by identifying plausible transitions between motion segments, and efficient search through the resulting graph structure is obtained through clustering. Three interface techniques are demonstrated for controlling avatar motion using this data structure: the user selects from a set of available choices, sketches a path through an environment, or acts out a desired motion in front of a video camera. We demonstrate the flexibility of the approach through four different applications and compare the avatar motion to directly recorded human motion.

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	Okan Arikan , D. A. Forsyth, Interactive motion generation from examples, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
	2	BADLER, N. I., HOLLICK, M., AND GRANIERI, J. 1993. Real-time control of a virtual human using minimal sensors. Presence 2, 82-86.
	3	Jezekiel Ben-Arie , Purvin Pandit , ShyamSundar Rajaram, Design of a digital library for human movement, Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries, p.300-309, January 2001, Roanoke, Virginia, United States [doi>10.1145/379437.379692]
	4	Bruce M. Blumberg , Tinsley A. Galyean, Multi-level direction of autonomous creatures for real-time virtual environments, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.47-54, September 1995 [doi>10.1145/218380.218405]
	5	Bruce M. Blumberg, Swamped! using plush toys to direct autonomous animated characters, ACM SIGGRAPH 98 Conference abstracts and applications, p.109, July 19-24, 1998, Orlando, Florida, United States [doi>10.1145/280953.281021]
	6	BOWDEN, R. 2000. Learning statistical models of human motion. In IEEE Workshop on Human Modelling, Analysis and Synthesis, CVPR2000.
	7	BRADLEY, E., AND STUART, J. 1997. Using chaos to generate choreographic variations. In Proceedings of the Experimental Chaos Conference.
	8	Matthew Brand , Aaron Hertzmann, Style machines, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.183-192, July 2000 [doi>10.1145/344779.344865]
	9	Matthew Brand, Shadow Puppetry, Proceedings of the International Conference on Computer Vision-Volume 2, p.1237, September 20-25, 1999
	10	A. Bruderlin , T. W. Calvert, Goal-directed, dynamic animation of human walking, ACM SIGGRAPH Computer Graphics, v.23 n.3, p.233-242, July 1989
	11	Armin Bruderlin , Tom Calvert, Knowledge-driven, interactive animation of human running, Proceedings of the conference on Graphics interface '96, p.213-221, May 1996, Toronto, Ontario, Canada
	12	Armin Bruderlin , Lance Williams, Motion signal processing, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.97-104, September 1995 [doi>10.1145/218380.218421]
	13	Justine Cassell , Hannes Högni Vilhjálmsson , Timothy Bickmore, BEAT: the Behavior Expression Animation Toolkit, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.477-486, August 2001 [doi>10.1145/383259.383315]
	14	Justine Cassell, Embodied conversational interface agents, Communications of the ACM, v.43 n.4, p.70-78, April 2000 [doi>10.1145/332051.332075]
	15	Diane Chi , Monica Costa , Liwei Zhao , Norman Badler, The EMOTE model for effort and shape, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.173-182, July 2000 [doi>10.1145/344779.352172]
	16	Sonu Chopra-Khullar , Norman I. Badler, Where to look? Automating attending behaviors of virtual human characters, Proceedings of the third annual conference on Autonomous Agents, p.16-23, April 1999, Seattle, Washington, United States [doi>10.1145/301136.301152]
	17	FALOUTSOS, P., VAN DE PANNE, M., AND TERZOPOULOS, D. 2001. The virtual stuntman: dynamic characters with a repertoire of autonomous motor skills. Computers & Graphics 25, 6 (December), 933-953.
	18	Petros Faloutsos , Michiel van de Panne , Demetri Terzopoulos, Composable controllers for physics-based character animation, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.251-260, August 2001 [doi>10.1145/383259.383287]
	19	FRALEY, C., AND RAFTERY, A. E. 1998. How many clusters? Which clustering method? Answers via model-based cluster analysis. Computer Journal 41, 8, 578-588.
	20	Aphrodite Galata , Neil Johnson , David Hogg, Learning variable-length Markov models of behavior, Computer Vision and Image Understanding, v.81 n.3, p.398-413, March 2001 [doi>10.1006/cviu.2000.0894]
	21	Michael Gleicher, Motion editing with spacetime constraints, Proceedings of the 1997 symposium on Interactive 3D graphics, p.139-ff., April 27-30, 1997, Providence, Rhode Island, United States [doi>10.1145/253284.253321]
	22	Michael Gleicher, Retargetting motion to new characters, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.33-42, July 1998 [doi>10.1145/280814.280820]
	23	Micheal Gleicher, Comparing constraint-based motion editing methods, Graphical Models, v.63 n.2, p.107-134, March 2001 [doi>10.1006/gmod.2001.0549]
	24	Jessica K. Hodgins , Wayne L. Wooten , David C. Brogan , James F. O'Brien, Animating human athletics, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.71-78, September 1995 [doi>10.1145/218380.218414]
	25	HU, M. K. 1962. Visual pattern recognition by moment invariants. IRE Transactions on Information Theory 8, 2, 179-187.
	26	Lucas Kovar , Michael Gleicher , Frédéric Pighin, Motion graphs, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
	27	Alexis Lamouret , Michiel van de Panne, Motion synthesis by example, Proceedings of the Eurographics workshop on Computer animation and simulation '96, p.199-212, December 1996, Poitiers, France
	28	Joseph Laszlo , Michiel van de Panne , Eugene Fiume, Limit cycle control and its application to the animation of balancing and walking, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.155-162, August 1996 [doi>10.1145/237170.237231]
	29	Jehee Lee , Sung Yong Shin, A hierarchical approach to interactive motion editing for human-like figures, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.39-48, July 1999 [doi>10.1145/311535.311539]
	30	Yan Li , Tianshu Wang , Heung-Yeung Shum, Motion texture: a two-level statistical model for character motion synthesis, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
	31	Tom Molet , Ronan Boulic , Daniel Thalmann, A real time anatomical converter for human motion capture, Proceedings of the Eurographics workshop on Computer animation and simulation '96, p.79-94, December 1996, Poitiers, France
	32	MOLET, T., AUBEL, T., GAPIN, T., CARION, S., AND LEE, E. 1999. Anyone for tennis? Presence 8, 2, 140-156.
	33	L. M. Tanco , A. Hilton, Realistic synthesis of novel human movements from a database of motion capture examples, Proceedings of the Workshop on Human Motion (HUMO'00), p.137, December 07-08, 2000
	34	OXFORD METRIC SYSTEMS, 2002. www.vicon.com.
	35	Ken Perlin , Athomas Goldberg, Improv: a system for scripting interactive actors in virtual worlds, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.205-216, August 1996 [doi>10.1145/237170.237258]
	36	Ken Perlin, Real Time Responsive Animation with Personality, IEEE Transactions on Visualization and Computer Graphics, v.1 n.1, p.5-15, March 1995 [doi>10.1109/2945.468392]
	37	Katherine Pullen , Christoph Bregler, Animating by Multi-Level Sampling, Proceedings of the Computer Animation, p.36, May 03-05, 2000
	38	Katherine Pullen , Christoph Bregler, Motion capture assisted animation: texturing and synthesis, Proceedings of the 29th annual conference on Computer graphics and interactive techniques, July 23-26, 2002, San Antonio, Texas
	39	ROSALES, R., ATHITSOS, V., SIGAL, L., AND SCLAROFF, S. 2001. 3D hand pose reconstruction using specialized mappings. In IEEE International Conference on Computer Vision, 378-385.
	40	Charles Rose , Michael F. Cohen , Bobby Bodenheimer, Verbs and Adverbs: Multidimensional Motion Interpolation, IEEE Computer Graphics and Applications, v.18 n.5, p.32-40, September 1998 [doi>10.1109/38.708559]
	41	SARCOS, 2002. www.sarcos.com.
	42	Arno Schödl , Richard Szeliski , David H. Salesin , Irfan Essa, Video textures, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.489-498, July 2000 [doi>10.1145/344779.345012]
	43	SEMWAL, S., HIGHTOWER, R., AND STANSFIELD, S. 1998. Mapping algorithms for real-time control of an avatar using eight sensors. Presence 7, 1, 1-21.
	44	Hedvig Sidenbladh , Michael J. Black , Leonid Sigal, Implicit Probabilistic Models of Human Motion for Synthesis and Tracking, Proceedings of the 7th European Conference on Computer Vision-Part I, p.784-800, May 28-31, 2002
	45	Harold C. Sun , Dimitris N. Metaxas, Automating gait generation, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.261-270, August 2001 [doi>10.1145/383259.383288]
	46	TARJAN, R. 1972. Depth first search and linear graph algorithms. SIAM Journal of Computing 1, 146-160.
	47	Munetoshi Unuma , Ken Anjyo , Ryozo Takeuchi, Fourier principles for emotion-based human figure animation, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.91-96, September 1995 [doi>10.1145/218380.218419]
	48	Douglas J. Wiley , James K. Hahn, Interpolation Synthesis of Articulated Figure Motion, IEEE Computer Graphics and Applications, v.17 n.6, p.39-45, November 1997 [doi>10.1109/38.626968]
	49	Andrew Witkin , Zoran Popovic, Motion warping, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.105-108, September 1995 [doi>10.1145/218380.218422]
	50	WOOTEN, W. L., AND HODGINS, J. K. 1996. Animation of human diving. Computer Graphics Forum 15, 1, 3-14.

Collaborative Colleagues:

Jehee Lee: colleagues

Jinxiang Chai: colleagues

Paul S. A. Reitsma: colleagues

Jessica K. Hodgins: colleagues

Nancy S. Pollard: colleagues