Latent topic driving model for movie affective scene classification

ABSTRACT

This paper proposes a latent topic driving model (LTDM) as a novel approach to movie affective scene classification. LTDM is a discriminative model of emotions driven by movie affective contents. Unlike existing methods, our approach is based on movie topic extraction via the latent Dirichlet allocation (LDA) and emotion dynamics modeling with reference to Plutchik's emotion theory. The classification procedure starts by segmenting movie scenes into movie shots, each of which is represented by a histogram of quantized affect-related audio-visual features. LDA is applied to detect topics of each movie shot. Emotions for the current movie shot are estimated based on both the topics of the shot and emotion transition weights determined by Plutchik's emotion theory. We conduct experiments using 206 movie scenes extracted from 24 movie titles (total 6 hours 20 min. 12 sec.) and the labels of eight emotion categories given by 16 subjects are collected. The results show that LTDM outperforms conventional modeling approaches in terms of the subject agreement rate.

REFERENCES

• 1: A. Hanjalic and L.-Q. Xu, "Affective Video Content Representation and Modeling," IEEE Trans. Multimedia, vol.7, no. 1, pp. 143--154, 2005.
• 2: H.B. Kang, "Affective Content Detection Using HMMs," Proc. ACM MM, pp.259--262, 2003.
• 3: M. Xu, L.-T. Chia and J. Jin, "Affective Content Analysis in Comedy and Horror Videos by Audio Emotional Event Detection," Proc. ICME, pp. 622--625, 2005.
• 4: S. Arifin and P.Y.K. Cheung, "Affective Level Video Segmentation by Utilizing the Pleasure-Arousal-Dominance Information," IEEE Trans. Multimedia, vol. 10, no. 7, pp.1325--1341, 2008.
• 5: M. Xu, J.S. Jin, S. Luo and L. Duan, "Hierarchical Movie Affective Content Analysis Based on Arousal and Valence Features," Proc. ACM MM, pp. 677--680, 2008.
• 6: D.M. Blei, A.Y. Ng, and M.I. Jordan, "Latent Dirichlet Allocation," Journal of Machine Learning Research, vol. 3, pp.993--1022, 2003.
• 7: R. Plutchik, "The Nature of Emotions," American Scientist, Vol. 89, No. 4, pp. 344--350, 2001.
• 8: (eds.) J.A. Coan and J.J.B. Allen, Handbook of Emotion Elicitation and Assessment, Oxford Univ. Press, 2007.
• 9: G. Csurka, C.R. Dance, L. Fan, J. Willamowski, and C. Bray, "Visual Categorization with Bags of Keypoints," Proc. ECCV, pp. 1--22, 2004.
• 10: F. Souvannavong, B. Merialdo, and B. Huet, "Latent Semantic Analysis For An Effective Region-Based Video Retrieval System," Proc. MIR, pp.243--250, 2004.
• 11: A. McCallum and F. Pereira, "Maximum Entropy Markov Models for Information Extraction and Segmentation," Proc. ICML, pp.591--598, 2000.
• 12: J.J. Gross and R.W. Levenson, "Emotion Elicitation Using Films," Cognition and Emotion, vol. 9, no. 1, pp.87--108, 1995.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

General Terms:
Algorithms, Design, Experimentation

Keywords:
Plutchik's basic emotions, affective scene classification, latent dirichlet allocation, latent topic driving model