Generating 3D views of facial expressions from frontal face video based on topographic analysis

ABSTRACT

In this paper, we report our newly developed 3D face modeling system with arbitrary expressions in a high level of detail using the topographic analysis and mesh instantiation process. Given a sequence of images of facial expressions at frontal views, we automatically generate 3D expressions at arbitrary views. Our face modeling system consists of two major components: facial surface representation using topographic analysis and generic model individualization based on labeled surface features and surface curvatures. The realism of the generated individual model is demonstrated through 3D views of facial expressions in videos. This work targets the accurate modeling of face and face expression for human computer interaction and 3D face recognition.

REFERENCES

• 1: Volker Blanz , Thomas Vetter, A morphable model for the synthesis of 3D faces, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.187-194, July 1999 [doi>10.1145/311535.311556]
• 2: T. F. Cootes and C. Taylor. Statistical models of appearance for computer vision. In Technical Report, University of Manchester, Manchester, UK, 2001.
• 3: L. Zalewski, et al. Synthesis and recognition of facial expressions in virtual 3d views. IEEE 6th Inter. Conf. on Automatic Face and Gesture Recognition, 2004.
• 4: R. Haralick and et al. The topographic primal sketch. The Int. J of Robotics Research, 2(2):50--72, 1983.
• 5: Frédéric Pighin , Jamie Hecker , Dani Lischinski , Richard Szeliski , David H. Salesin, Synthesizing realistic facial expressions from photographs, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.75-84, July 1998 [doi>10.1145/280814.280825]
• 6: D. Terzopoulos , K. Waters, Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.15 n.6, p.569-579, June 1993 [doi>10.1109/34.216726]
• 7: Øivind Due Trier , Torfinn Taxt , Anil K. Jain, Recognition of Digits in Hydrographic Maps: Binary Versus Topographic Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.4, p.399-404, April 1997 [doi>10.1109/34.588025]
• 8: Lijun Yin , Anup Basu, Generating realistic facial expressions with wrinkles for model-based coding, Computer Vision and Image Understanding, v.84 n.2, p.201-240, November 2001 [doi>10.1006/cviu.2001.0949]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.7 Feature Measurement
          Subjects: Feature representation

Additional Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
  I.5 PATTERN RECOGNITION
      I.5.4 Applications

General Terms:
Algorithms

Keywords:
face modeling, facial expression, feature analysis