Data-driven enhancement of facial attractiveness

ABSTRACT

When human raters are presented with a collection of shapes and asked to rank them according to their aesthetic appeal, the results often indicate that there is a statistical consensus among the raters. Yet it might be difficult to define a succinct set of rules that capture the aesthetic preferences of the raters. In this work, we explore a data-driven approach to aesthetic enhancement of such shapes. Specifically, we focus on the challenging problem of enhancing the aesthetic appeal (or the attractiveness) of human faces in frontal photographs (portraits), while maintaining close similarity with the original.

The key component in our approach is an automatic facial attractiveness engine trained on datasets of faces with accompanying facial attractiveness ratings collected from groups of human raters. Given a new face, we extract a set of distances between a variety of facial feature locations, which define a point in a high-dimensional "face space". We then search the face space for a nearby point with a higher predicted attractiveness rating. Once such a point is found, the corresponding facial distances are embedded in the plane and serve as a target to define a 2D warp field which maps the original facial features to their adjusted locations. The effectiveness of our technique was experimentally validated by independent rating experiments, which indicate that it is indeed capable of increasing the facial attractiveness of most portraits that we have experimented with.

REFERENCES

• 1: Alley, T. R., and Cunningham, M. R. 1991. Averaged faces are attractive, but very attractive faces are not average. Psychological Science 2, 123--125.
• 2: Thaddeus Beier , Shawn Neely, Feature-based image metamorphosis, Proceedings of the 19th annual conference on Computer graphics and interactive techniques, p.35-42, July 1992
• 3: Volker Blanz , Thomas Vetter, A morphable model for the synthesis of 3D faces, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.187-194, July 1999 [doi>10.1145/311535.311556]
• 4: Blanz, V., 2003. Manipulation of facial attractiveness. {web page} http://www.mpi-inf.mpg.de/~blanz/data/attractiveness/.
• 5: Bradski, G. 2000. The OpenCV library. Dr. Dobbs Journal 25, 11 (Nov.), 120, 122--125.
• 6: Jonathan D. Cohen, Drawing graphs to convey proximity: an incremental arrangement method, ACM Transactions on Computer-Human Interaction (TOCHI), v.4 n.3, p.197-229, Sept. 1997 [doi>10.1145/264645.264657]
• 7: T. F. Cootes , C. J. Taylor , D. H. Cooper , J. Graham, Active shape models—their training and application, Computer Vision and Image Understanding, v.61 n.1, p.38-59, Jan. 1995 [doi>10.1006/cviu.1995.1004]
• 8: Cunningham, M. R., Roberts, A. R., Wu, C.-H., Barbee, A. P., and Druen, P. B. 1995. Their ideas of beauty are, on the whole, the same as ours: Consistency and variability in the cross-cultural perception of female attractiveness. Journal of Personality and Social Psychology 68, 1, 261--279.
• 9: Yael Eisenthal , Gideon Dror , Eytan Ruppin, Facial Attractiveness: Beauty and the Machine, Neural Computation, v.18 n.1, p.119-142, January 2006 [doi>10.1162/089976606774841602]
• 10: Brian Guenter , Cindy Grimm , Daniel Wood , Henrique Malvar , Fredric Pighin, Making faces, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.55-66, July 1998 [doi>10.1145/280814.280822]
• 11: Thorsten Joachims, Making large-scale support vector machine learning practical, Advances in kernel methods: support vector learning, MIT Press, Cambridge, MA, 1999
• 12: Jones, D. 1996. Physical Attractiveness and the Theory of Sexual Selection: Results from Five Populations. University of Michigan Press, Ann Arbor, Michigan.
• 13: Kagian, A., Dror, G., Leyvand, T., Cohen-Or, D., and Ruppin, E. 2007. A humanlike predictor of facial attractiveness. In Advances in Neural Information Processing Systems 19, B. Schölkopf, J. Platt, and T. Hoffman, Eds. MIT Press.
• 14: Langlois, J. H., Roggman, L. A., Casey, R. J., M. Ritter, J., Rieser-Danner, L. A., and Jenkins, V. Y. 1987. Infant preferences for attractive faces: Rudiments of a stereotype? Developmental Psychology 23, 5, 363--369.
• 15: A. Lanitis , C. J. Taylor , T. F. Cootes, Toward Automatic Simulation of Aging Effects on Face Images, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.4, p.442-455, April 2002 [doi>10.1109/34.993553]
• 16: Yuencheng Lee , Demetri Terzopoulos , Keith Walters, Realistic modeling for facial animation, Proceedings of the 22nd annual conference on Computer graphics and interactive techniques, p.55-62, September 1995 [doi>10.1145/218380.218407]
• 17: Seungyong Lee , George Woberg , Kyung-Yong Chwa , Sung Yong Shin, Image Metamorphosis with Scattered Feature Constraints, IEEE Transactions on Visualization and Computer Graphics, v.2 n.4, p.337-354, December 1996 [doi>10.1109/2945.556502]
• 18: Seungyong Lee , George Wolberg , Sung Yong Shin, Scattered Data Interpolation with Multilevel B-Splines, IEEE Transactions on Visualization and Computer Graphics, v.3 n.3, p.228-244, July 1997 [doi>10.1109/2945.620490]
• 19: Levenberg, K. 1944. A method for the solution of certain problems in least squares. Quart. Appl. Math., 2, 164--168.
• 20: Lourakis, M., 2004. levmar: Levenberg-Marquardt non-linear least squares algorithms in C/C++. {web page} http://www.ics.forth.gr/~lourakis/levmar/.
• 21: Marquardt, D. 1963. An algorithm for least-squares estimation of nonlinear parameters. SIAM J. Appl. Math., 11, 431--441.
• 22: Martinez, A. M., and Benavente, R. 1998. The AR face database. CVC Technical Report 24.
• 23: O'Toole, A. J., Price, T., Vetter, T., Bartlett, J. C., and Blanz, V. 1999. 3D shape and 2D surface textures of human faces: the role of "averages" in attractiveness and age. Image Vision Comput. 18, 1, 9--19.
• 24: Frederic I. Parke , Keith Waters, Computer facial animation, A. K. Peters, Ltd., Natick, MA, 1996
• 25: Patrick Pérez , Michel Gangnet , Andrew Blake, Poisson image editing, ACM Transactions on Graphics (TOG), v.22 n.3, July 2003
• 26: Perrett, D. I., May, K. A., and Yoshikawa, S. 1994. Facial shape and judgements of female attractiveness. Nature 368, 239--242.
• 27: Perrett, D. I., Burt, D. M., Penton-Voak, I. S., Lee, K. J., Rowland, D. A., and Edwards, R. 1999. Symmetry and human facial attractiveness. Evolution and Human Behavior 20, 295--307.
• 28: Frédéric Pighin , Jamie Hecker , Dani Lischinski , Richard Szeliski , David H. Salesin, Synthesizing realistic facial expressions from photographs, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.75-84, July 1998 [doi>10.1145/280814.280825]
• 29: William H. Press , Saul A. Teukolsky , William T. Vetterling , Brian P. Flannery, Numerical recipes in C (2nd ed.): the art of scientific computing, Cambridge University Press, New York, NY, 1992
• 30: J. W. Sammon, A Nonlinear Mapping for Data Structure Analysis, IEEE Transactions on Computers, v.18 n.5, p.401-409, May 1969 [doi>10.1109/T-C.1969.222678]
• 31: Slater, A., Der Schulenberg, C. V., Brown, E., Butter-worth, G., Badenoch, M., and Parsons, S. 1998. Newborn infants prefer attractive faces. Infant Behavior and Development, 345--354.
• 32: Vladimir N. Vapnik, The nature of statistical learning theory, Springer-Verlag New York, Inc., New York, NY, 1995
• 33: Ming-Hsuan Yang , David J. Kriegman , Narendra Ahuja, Detecting Faces in Images: A Survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.1, p.34-58, January 2002 [doi>10.1109/34.982883]
• 34: W. Zhao , R. Chellappa , P. J. Phillips , A. Rosenfeld, Face recognition: A literature survey, ACM Computing Surveys (CSUR), v.35 n.4, p.399-458, December 2003 [doi>10.1145/954339.954342]
• 35: Zhou, Y., Gu, L., and Zhang, H.-J. 2003. Bayesian tangent shape model: Estimating shape and pose parameters via bayesian inference. In Proc. CVPR 2003, 109--118.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.3 Enhancement
          Subjects: Geometric correction

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.1 Digitization and Image Capture
          Subjects: Imaging geometry
      I.4.7 Feature Measurement
          Subjects: Size and shape
  I.6 SIMULATION AND MODELING
      I.6.4 Model Validation and Analysis

General Terms:
Algorithms, Design, Measurement, Performance

Keywords:
facial attractiveness, machine learning, optimization, warping