Latent semantic analysis of facial action codes for automatic facial expression recognition

ABSTRACT

For supervised training of automatic facial expression recognition systems, adequate ground truth labels that describe relevant facial expression categories are necessary. One possibility is to label facial expressions into emotion categories. Another approach is to label facial expressions independently from any interpretation attempts. This can be achieved via the facial action coding system (FACS). In this paper we present a novel approach that allows to automatically cluster FACScodes into meaningful categories. Our approach exploits the fact that FACScodes can be seen as documents containing terms -the action units (AUs) present in the codes-and so text modeling methods that capture co-occurrence information in low-dimensional spaces can be used. The FACScode derived descriptions are computed by Latent Semantic Analysis (LSA) and Probabilistic Latent Semantic Analysis (PLSA). We show that, as a high-level description of facial actions, the newly derived codes constitute a competitive alternative to both basic emotion and FACScodes. We have used them to train different types of artificial neural networks

REFERENCES

• 1: Ricardo A. Baeza-Yates , Berthier Ribeiro-Neto, Modern Information Retrieval, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1999
• 2: M. Bartlett, G. Littlewort, I. Fasel, and J. Movellan. Real Time Face Detection and Expression Recognition: Development and Application to Human-Computer Interaction. In CVPR Workshop on Computer Vision and Pattern Recognition for Human-Computer Interaction, 2003.
• 3: S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman. Indexing by Latent Semantic Analysis. Journal of the American Society for Information Science, 41, 1990.
• 4: P. Ekman. Methods for Measuring Facial Actions. In K. Scherer and P. Ekman, editors, Handbook of Methods in Nonverbal Behaviour Research, pages 45--90. Cambridge University, 1982.
• 5: P. Ekman and W. Friesen. Constants Across Cultures in the Face and Emotion. Journal of Personality and Social Psychology, 17(2):124--129, 1971.
• 6: P. Ekman and W. Friesen. Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto, 1978.
• 7: M. Lyons , S. Akamatsu , M. Kamachi , J. Gyoba, Coding Facial Expressions with Gabor Wavelets, Proceedings of the 3rd. International Conference on Face & Gesture Recognition, p.200, April 14-16, 1998
• 8: P. Ekman, W. V. Friesen, and J. C. Hager. FACS Investigator's Guide. A Human Face, 666 Malibu Drive, Salt Lake City UT 84107, 2002.
• 9: B. Fasel. Robust Face Analysis using Convolutional Neural Networks. In Proceedings of the International Conference on Pattern Recognition (ICPR 02), volume 2, pages 40--43, Quebec, Canada, aug 2002. IDIAP-RR 01-48.
• 10: B. Fasel and J. Luettin. Automatic Facial Expression Analysis: A Survey. Pattern Recognition, 36(1):259--275, 2003. IDIAP-RR 99-19.
• 11: Florent Monay , Daniel Gatica-Perez, On image auto-annotation with latent space models, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957070]
• 12: T. Hofmann. Probabilistic Latent Semantic Analysis. In Proceedings of Uncertainty in Artificial Intelligence (UAI 99), Stockholm, Sweden, 1999.
• 13: Takeo Kanade , Yingli Tian , Jeffrey F. Cohn, Comprehensive Database for Facial Expression Analysis, Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000, p.46, March 26-30, 2000
• 14: Yann LeCun , Yoshua Bengio, Convolutional networks for images, speech, and time series, The handbook of brain theory and neural networks, MIT Press, Cambridge, MA, 1998
• 15: Maja Pantic , Leon J. M. Rothkrantz, Automatic Analysis of Facial Expressions: The State of the Art, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.12, p.1424-1445, December 2000 [doi>10.1109/34.895976]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

General Terms:
Algorithms, Theory

Keywords:
automatic facial expression recognition, latent semantic analysis