Face poser: interactive modeling of 3D facial expressions using model priors

ABSTRACT

In this paper, we present an intuitive interface for interactively posing 3D facial expressions. The user can create and edit facial expressions by drawing freeform strokes, or by directly dragging facial points in 2D screen space. Designing such an interface for face modeling and editing is challenging because many unnatural facial expressions might be consistent with the ambiguous user input. The system automatically learns a model prior from a prerecorded facial expression database and uses it to remove the ambiguity. We formulate the problem in a maximum a posteriori (MAP) framework by combining the prior with user-defined constraints. Maximizing the posterior allows us to generate an optimal and natural facial expression that satisfies the user-defined constraints. Our system is interactive; it is also simple and easy to use. A first-time user can learn to use the system and start creating a variety of natural face models within minutes. We evaluate the performance of our approach with cross validation tests, and by comparing with alternative techniques.

REFERENCES

• 1: Christopher G. Atkeson , Andrew W. Moore , Stefan Schaal, Locally Weighted Learning, Artificial Intelligence Review, v.11 n.1-5, p.11-73, Feb. 1997 [doi>10.1023/A:1006559212014]
• 2: {BBPV03} Blanz V., Basso C., Poggio T., Vetter T.: Reanimating faces in images and video. In Computer Graphics Forum (2003). 22(3):641--650.
• 3: Christopher M. Bishop, Neural networks for pattern recognition, Oxford University Press, Oxford, 1996
• 4: Matthew Brand, Incremental Singular Value Decomposition of Uncertain Data with Missing Values, Proceedings of the 7th European Conference on Computer Vision-Part I, p.707-720, May 28-31, 2002
• 5: Volker Blanz , Thomas Vetter, A morphable model for the synthesis of 3D faces, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.187-194, July 1999 [doi>10.1145/311535.311556]
• 6: Christopher M. Bishop , John M. Winn, Non-linear Bayesian Image Modelling, Proceedings of the 6th European Conference on Computer Vision-Part I, p.3-17, June 26-July 01, 2000
• 7: Jinxiang Chai , Jessica K. Hodgins, Performance animation from low-dimensional control signals, ACM Transactions on Graphics (TOG), v.24 n.3, July 2005
• 8: Edwin Chang , Odest Chadwicke Jenkins, Sketching articulation and pose for facial animation, Proceedings of the 2006 ACM SIGGRAPH/Eurographics symposium on Computer animation, September 02-04, 2006, Vienna, Austria
• 9: Jin-xiang Chai , Jing Xiao , Jessica Hodgins, Vision-based control of 3D facial animation, Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 26-27, 2003, San Diego, California
• 10: {DH73} Duda R., Hart P.: Pattern Classification and Scene Analysis. John Wiley and Sons, 1973.
• 11: Kevin G. Der , Robert W. Sumner , Jovan Popović, Inverse kinematics for reduced deformable models, ACM Transactions on Graphics (TOG), v.25 n.3, July 2006
• 12: Brian Gough, GNU Scientific Library Reference Manual - 2nd Edition, Network Theory Ltd., 2003
• 13: {GH97} Ghahramani Z., Hinton G. E.: The EM algorithm for mixtures of factor analyzers, 1997.
• 14: Keith Grochow , Steven L. Martin , Aaron Hertzmann , Zoran Popović, Style-based inverse kinematics, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004
• 15: {HDR97} Hinton G. E., Dayan P., Revow M.: Modeling the manifolds of images of handwritten digits. In IEEE Transactions on Neural Networks. 1997. 8(1):65--74.
• 16: Takeo Igarashi , Satoshi Matsuoka , Hidehiko Tanaka, Teddy: a sketching interface for 3D freeform design, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.409-416, July 1999 [doi>10.1145/311535.311602]
• 17: Pushkar Joshi , Wen C. Tien , Mathieu Desbrun , Frédéric Pighin, Learning controls for blend shape based realistic facial animation, Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 26-27, 2003, San Diego, California
• 18: Youngihn Kho , Michael Garland, Sketching mesh deformations, Proceedings of the 2005 symposium on Interactive 3D graphics and games, April 03-06, 2005, Washington, District of Columbia [doi>10.1145/1053427.1053452]
• 19: {Law04} Lawrence N. D.: Gaussian process latent variable models for visualization of high dimensional data. In Advances in Neural Information Processing Systems 16 (2004). 329--336.
• 20: J. P. Lewis , Matt Cordner , Nickson Fong, Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.165-172, July 2000 [doi>10.1145/344779.344862]
• 21: Jun-yong Noh , Ulrich Neumann, Expression cloning, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.277-288, August 2001 [doi>10.1145/383259.383290]
• 22: Andrew Nealen , Olga Sorkine , Marc Alexa , Daniel Cohen-Or, A sketch-based interface for detail-preserving mesh editing, ACM Transactions on Graphics (TOG), v.24 n.3, July 2005
• 23: Frederick I. Parke, Computer generated animation of faces, Proceedings of the ACM annual conference, August 01-01, 1972, Boston, Massachusetts, United States [doi>10.1145/800193.569955]
• 24: {PSS99} Pighin F., Szeliski R., Salesin D.: Resynthesizing facial animation through 3d model-based tracking. In International Conference on Computer Vision. 1999. 143--150.
• 25: Frederic I. Parke , Keith Waters, Computer facial animation, A. K. Peters, Ltd., Natick, MA, 1996
• 26: Peter-Pike J. Sloan , Charles F. Rose, III , Michael F. Cohen, Shape by example, Proceedings of the 2001 symposium on Interactive 3D graphics, p.135-143, March 2001 [doi>10.1145/364338.364382]
• 27: Robert W. Sumner , Matthias Zwicker , Craig Gotsman , Jovan Popović, Mesh-based inverse kinematics, ACM Transactions on Graphics (TOG), v.24 n.3, July 2005
• 28: {YSvdP05} Yang C., Sharon D., van de Panne M.: Sketch-based modeling of parameterized objects. In 2nd Eurographics Workshop on Sketch-Based Interfaces and Modeling (2005).
• 29: Robert C. Zeleznik , Kenneth P. Herndon , John F. Hughes, SKETCH: an interface for sketching 3D scenes, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.163-170, August 1996 [doi>10.1145/237170.237238]
• 30: Qingshan Zhang , Zicheng Liu , Baining Guo , Harry Shum, Geometry-driven photorealistic facial expression synthesis, Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 26-27, 2003, San Diego, California
• 31: Li Zhang , Noah Snavely , Brian Curless , Steven M. Seitz, Spacetime faces: high resolution capture for modeling and animation, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.5 Computational Geometry and Object Modeling

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.6 Methodology and Techniques