Analysis of scene context related with emotional events

ABSTRACT

We propose a method for the detection of emotional events by analyzing scene context in video data. To analyze scene-level features, we compute the contextual information in video scenes. We divide the scene context into intra-scene context and interscene context. We define intra-scene context as the context within the scene such as the shots' coherences, shot's interactions and dominant features in color and motion information within the scene. We also define the inter-scene context as the given scene's relationship with other scenes. In detecting emotional events from video data, we first compute inter-scene context. Then, we select possible candidates which have large dissimilarities in comparison with previous scenes. Finally, we make a decision whether the candidate is emotional event or not. We experiment our proposed approach on test data and detect 70% of the occurrences of the emotional events such as fear and sadness.

REFERENCES

• 1: A. Hanjalic, "Video and Image Retrieval beyond the Cognitive Level: The Needs and Possibilities," Proc. SPIE Storage and Retrieval for Media Databases 2001, San Jose, CA, pp. 130--140, 2001.
• 2: Rosalind W. Picard, Affective computing, MIT Press, Cambridge, MA, 1997
• 3: Rosalind W. Picard , Elias Vyzas , Jennifer Healey, Toward Machine Emotional Intelligence: Analysis of Affective Physiological State, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.10, p.1175-1191, October 2001 [doi>10.1109/34.954607]
• 4: Alan Hanjalic , Li-Qun Xu, User-oriented Affective Video Content Analysis, Proceedings of the IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL'01), p.50, December 14-14, 2001
• 5: Simon Moncrieff , Chitra Dorai , Svetha Venkatesh, Affect computing in film through sound energy dynamics, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500231]
• 6: Hari Sundaram , Shih-Fu Chang, Determining computable scenes in films and their structures using audio-visual memory models, Proceedings of the eighth ACM international conference on Multimedia, p.95-104, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.354440]
• 7: N. Dimitrova, J. Martino, L. Agnihotri and H. Elenbaas, "Color Superhistograms for Video Representation," Proc. ICIP '99.
• 8: E. Goldstein, Sensation and Perception, Brooks/Cole, 1999.
• 9: J. Itten, The Art of Color, 1973, Wiley.
• 10: B. Lucas, T. Kanade, "An Iterative Technique of Image registration and Its Application to Stereo," Proc. IJACI, pp. 674--679, 1981.
• 11: H. Zhang, J. Wu, D. Zhong, and S. Smoliar, "An integrated system for content-based video retrieval and browsing." Pattern Recognition, Vol. 30, No. 4, pp. 643--58, 1997.
• 12: Hang-Bong Kang, A Hierarchiacal Approach to Scene Segmentation, Proceedings of the IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL'01), p.65, December 14-14, 2001

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing

Keywords:
content analysis, emotional event, scene context