Computer puppetry: An importance-based approach

ABSTRACT

Computer puppetry maps the movements of a performer to an animated character in real-time. In this article, we provide a comprehensive solution to the problem of transferring the observations of the motion capture sensors to an animated character whose size and proportion may be different from the performer's. Our goal is to map as many of the important aspects of the motion to the target character as possible, while meeting the online, real-time demands of computer puppetry. We adopt a Kalman filter scheme that addresses motion capture noise issues in this setting. We provide the notion of dynamic importance of an end-effector that allows us to determine what aspects of the performance must be kept in the resulting motion. We introduce a novel inverse kinematics solver that realizes these important aspects within tight real-time constraints. Our approach is demonstrated by its application to broadcast television performances.

REFERENCES

• 1: Ascension Technology Corp. 1996. Motion Star Installation and Operation Guide. Ascension Technology Corporation, Burlington, Vt.
• 2: Ali Azarbayejani , Alex P. Pentland, Recursive Estimation of Motion, Structure, and Focal Length, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.17 n.6, p.562-575, June 1995 [doi>10.1109/34.387503]
• 3: Ronald Azuma , Gary Bishop, Improving static and dynamic registration in an optical see-through HMD, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.197-204, July 1994 [doi>10.1145/192161.192199]
• 4: BADLER, N., HOLLICK,M.J.,AND GRANIERI, J. P. 1993. Real-time control of a virtual human using mininal sensors. Presence 2, 1, 82-86.
• 5: Rama Bindiganavale , Norman I. Badler, Motion Abstraction and Mapping with Spatial Constraints, Proceedings of the International Workshop on Modelling and Motion Capture Techniques for Virtual Environments, p.70-82, January 1998
• 6: BODENHEIMER, B., ROSE, C., ROSENTHAL,S.,AND PELLA, J. 1997. The process of motion capture: Dealing with the data. In Proceedings of the Eurographics Workshop on Computer Animation and Simulation'97. Springer-Verlag, Wein, Austria.
• 7: T J Broida , R Chellappa, Estimation of object motion parameters from noisy images, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.8 n.1, p.90-99, Jan. 1986 [doi>10.1109/TPAMI.1986.4767755]
• 8: CHOI, K.-J. AND KO, H.-S. 2000. On-line motion retargetting. J. Visual. Comput. Animation 11, 223-243.
• 9: Martin Friedmann , Thad Starner , Alex Pentland, Synchronization in virtual realities, Presence: Teleoperators and Virtual Environments, v.1 n.1, p.139-144, Winter 1992
• 10: Michael Girard , A. A. Maciejewski, Computational modeling for the computer animation of legged figures, Proceedings of the 12th annual conference on Computer graphics and interactive techniques, p.263-270, July 1985
• 11: Michael Gleicher, Motion editing with spacetime constraints, Proceedings of the 1997 symposium on Interactive 3D graphics, p.139-ff., April 27-30, 1997, Providence, Rhode Island, United States [doi>10.1145/253284.253321]
• 12: Michael Gleicher, Retargetting motion to new characters, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.33-42, July 1998 [doi>10.1145/280814.280820]
• 13: GULLAPALLI, V., GELFAND,J.J.,LANE,S.H.,AND WILSON, W. W. 1996. Synergy-based learning of hybrid position/force control for redundant manipulators. In Proceedings of the 1996 IEEE International Conference on Robotics and Automation. IEEE Computer Society Press, Los Alamitos, Calif.
• 14: HANSON, C. 2000. Scheme. http://www-swiss.ai.mit.edu/projects/scheme/.
• 15: Myoung-Jun Kim , Myung-Soo Kim , Sung Yong Shin, A general construction scheme for unit quaternion curves with simple high order derivatives, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.369-376, September 1995 [doi>10.1145/218380.218486]
• 16: Yoshihito Koga , Koichi Kondo , James Kuffner , Jean-Claude Latombe, Planning motions with intentions, Proceedings of the 21st annual conference on Computer graphics and interactive techniques, p.395-408, July 1994 [doi>10.1145/192161.192266]
• 17: KOREIN,J.U.AND BADLER, N. I. 1982. Techniques for generating the goal-directed motion of articulated structures. IEEE Comput. Graph. Appl. 2, 71-81.
• 18: Jehee Lee , Sung Yong Shin, A hierarchical approach to interactive motion editing for human-like figures, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.39-48, July 1999 [doi>10.1145/311535.311539]
• 19: Tom Molet , Ronan Boulic , Daniel Thalmann, A real time anatomical converter for human motion capture, Proceedings of the Eurographics workshop on Computer animation and simulation '96, p.79-94, December 1996, Poitiers, France
• 20: MOLET, T., BOULIC, R., AND THALMANN, D. 1999. Human motion capture driven by orientation measurements. Presence 8, 2, 187-203.
• 21: PADEN, B. 1986. Kinematics and control robot manipulators. Ph.D dissertation, University of California, Berkeley, Berkeley, Calif.
• 22: Zoran Popović , Andrew Witkin, Physically based motion transformation, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.11-20, July 1999 [doi>10.1145/311535.311536]
• 23: Protozoa. 1999. Technology information. http://www.protozoa.com/Page 2/info index.html.
• 24: Charles Rose , Brian Guenter , Bobby Bodenheimer , Michael F. Cohen, Efficient generation of motion transitions using spacetime constraints, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.147-154, August 1996 [doi>10.1145/237170.237229]
• 25: David J. Sturman, Computer Puppetry, IEEE Computer Graphics and Applications, v.18 n.1, p.38-45, January 1998 [doi>10.1109/38.637269]
• 26: TOLANI,D.AND BADLER, N. I. 1996. Real-time inverse kinematics of the human arm. Presence 5, 4, 393-401.
• 27: Deepak Tolani , Ambarish Goswami , Norman I. Badler, Real-time inverse kinematics techniques for anthropomorphic limbs, Graphical Models and Image Processing, v.62 n.5, p.353-388, Sept. 2000
• 28: Greg Welch , Gary Bishop, An Introduction to the Kalman Filter, University of North Carolina at Chapel Hill, Chapel Hill, NC, 1995
• 29: Greg Welch , Gary Bishop, SCAAT: incremental tracking with incomplete information, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.333-344, August 1997 [doi>10.1145/258734.258876]
• 30: WHITNEY, D. J. 1969. Resolved motion rate control of manipulators and human prostheses. IEEE Trans. Man-Mach. Syst. 47-53.
• 31: Jianmin Zhao , Norman I. Badler, Inverse kinematics positioning using nonlinear programming for highly articulated figures, ACM Transactions on Graphics (TOG), v.13 n.4, p.313-336, Oct. 1994 [doi>10.1145/195826.195827]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Animation

General Terms:
Algorithms

Keywords:
Human-figure animation, motion retargetting, performance-based animation, real-time animation