Expressive expression mapping with ratio images

ABSTRACT

Facial expressions exhibit not only facial feature motions, but also subtle changes in illumination and appearance (e.g., facial creases and wrinkles). These details are important visual cues, but they are difficult to synthesize. Traditional expression mapping techniques consider feature motions while the details in illumination changes are ignored. In this paper, we present a novel technique for facial expression mapping. We capture the illumination change of one person's expression in what we call an expression ratio image (ERI). Together with geometric warping, we map an ERI to any other person's face image to generate more expressive facial expressions.

REFERENCES

• 1: Stephen M. Platt , Norman I. Badler, Animating facial expressions, Proceedings of the 8th annual conference on Computer graphics and interactive techniques, p.245-252, August 03-07, 1981, Dallas, Texas, United States
• 2: Thaddeus Beier , Shawn Neely, Feature-based image metamorphosis, Proceedings of the 19th annual conference on Computer graphics and interactive techniques, p.35-42, July 1992
• 3: Christoph Bregler , Michele Covell , Malcolm Slaney, Video Rewrite: driving visual speech with audio, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.353-360, August 1997 [doi>10.1145/258734.258880]
• 4: S.E.Brennan.Caricature Generator .M.S.Visual Studies,Dept of Architecture,Massachusetts Institute of Technology,Cambridge,MA.,Sept.1982.
• 5: Paul Debevec, Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.189-198, July 1998 [doi>10.1145/280814.280864]
• 6: Paul Debevec , Tim Hawkins , Chris Tchou , Haarm-Pieter Duiker , Westley Sarokin , Mark Sagar, Acquiring the reflectance field of a human face, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.145-156, July 2000 [doi>10.1145/344779.344855]
• 7: James D. Foley , Andries van Dam , Steven K. Feiner , John F. Hughes, Computer graphics (2nd ed. in C): principles and practice, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1995
• 8: Rafael C. Gonzales , Paul Wintz, Digital image processing (2nd ed.), Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1987
• 9: Yuencheng Lee , Demetri Terzopoulos , Keith Walters, Realistic modeling for facial animation, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.55-62, September 1995 [doi>10.1145/218380.218407]
• 10: Lance Williams, Performance-driven facial animation, Proceedings of the 17th annual conference on Computer graphics and interactive techniques, p.235-242, September 1990, Dallas, TX, USA
• 11: S.R.Marschner and D.P.Greenberg.Inverse lighting for photography.In IST/SID Fifth Colort Imaging Conference ,November 1997.
• 12: Stephen R. Marschner , Brian K. Guenter , Sashi Raghupathy, Modeling and Rendering for Realistic Facial Animation, Proceedings of the Eurographics Workshop on Rendering Techniques 2000, p.231-242, June 26-28, 2000
• 13: Frederic I. Parke , Keith Waters, Computer facial animation, A. K. Peters, Ltd., Natick, MA, 1996
• 14: Frédéric Pighin , Jamie Hecker , Dani Lischinski , Richard Szeliski , David H. Salesin, Synthesizing realistic facial expressions from photographs, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.75-84, July 1998 [doi>10.1145/280814.280825]
• 15: T.Riklin-Raviv and A.Shashua.The quotient image:Class based re-rendering and recongnition withvarying illuminations. In IEEE Conference on Computer Vision and Pattern Recognition ,pages 566 -571,June 1999.
• 16: Steven M. Seitz , Charles R. Dyer, View morphing, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.21-30, August 1996 [doi>10.1145/237170.237196]
• 17: A.Stoschek.Image-based re-rendering of faces for continuous pose and illumination directions.In IEEE Conference on Computer Vision and Pattern Recognition ,pages 582 -587,2000.
• 18: D.Terzopoulos and K.Waters.Physically-based facial modeling and animation.Journal of Visualization and Computer Animation ,1(4):73 -80,March1990.
• 19: Keith Waters, A muscle model for animation three-dimensional facial expression, ACM SIGGRAPH Computer Graphics, v.21 n.4, p.17-24, July 1987
• 20: Lance Williams, Performance-driven facial animation, Proceedings of the 17th annual conference on Computer graphics and interactive techniques, p.235-242, September 1990, Dallas, TX, USA
• 21: George Wolberg, Digital Image Warping, IEEE Computer Society Press, Los Alamitos, CA, 1994

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.3 Picture/Image Generation
          Subjects: Display algorithms

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS

General Terms:
Algorithms, Design

Keywords:
animation, facial animation, morphing