Adaptive, selective, automatic tonal enhancement of faces

ABSTRACT

This paper presents an efficient, personalizable and yet completely automatic algorithm for enhancing the brightness, tonal balance, and contrast of faces in thumbnails of online videos where multiple colored illumination sources are the norm and artifacts such as poor illumination and backlight are common. These artifacts significantly lower the perceptual quality of faces and skin, and cannot be easily corrected by common global image transforms. The same identifiable user, however, often uploads or participates in multiple photos, videos, or video chat sessions with varying illumination conditions. The proposed algorithm adaptively transforms the skin pixels in a poor illumination environment to match the skin color model of a prototypical face of the same user in a better illumination environment. It leaves the remaining non-skin portions of the image virtually unchanged while ascertaining a smooth, natural appearance. A component of our system automatically selects such a prototypical face for each user given a collection of uploaded videos/photo albums or prior video chat sessions by that user. We present several human rating studies on YouTube data that quantitatively demonstrate significant improvement in facial quality using the proposed algorithm.

REFERENCES

• 1: E. Hsu, T. Mertens, S. Paris, S. Avidan, and F. Durand. Light mixture estimation for spatially varying white balance. ACM Trans. Graph., 27(3), 2008.
• 2: M. J. Jones and J. M. Rehg. Statistical color models with application to skin detection. IJCV, 46(1), 2002.
• 3: N. Joshi. Ph.D. Dissertation: Enhancing Photographs using Content-Specific Image Priors. 2008.
• 4: D. Lischinski, Z. Farbman, M. Uyttendaele, and R. Szeliski. Interactive local adjustment of tonal values. In SIGGRAPH, 2006.
• 5: E. Reinhard, M. Ashikhmin, B. Gooch, and P. Shirley. Color transfer between images. IEEE Computer Graphics and Applications, 21(5), 2001.
• 6: X. Xiao and L. Ma. Color transfer in correlated color space. In Proc. ACM Int. Conf. on Virtual reality continuum and its applications, 2006.
• 7: L. Yin, J. Jia, and J. Morrissey. Towards race-related face identification: Research on skin color transfer. Automatic Face and Gesture Recognition, IEEE Int. Conf. on, 0, 2004.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.3 Enhancement
          Subjects: Filtering

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.3 Enhancement
          Subjects: Grayscale manipulation; Smoothing

General Terms:
Algorithms

Keywords:
color transfer, facial enhancement, image quality