Affective content detection using HMMs

ABSTRACT

This paper discusses a new technique for detecting affective events using Hidden Markov Models(HMM). To map low level features of video data to high level emotional events, we perform empirical study on the relationship between emotional events and low-level features. After that, we compute simple low-level features that represent emotional characteristics and construct a token or observation vector by combining low level features. The observation vector sequence is tested to detect emotional events through HMMs. We create two HMM topologies and test both topologies. The affective events are detected from our proposed models with good accuracy.

REFERENCES

• 1: Hanjalic, A. Video and Image Retrieval beyond the Cognitive Level: The Needs and Possibilities. Proc. SPIE Storage and Retrieval for Media Databases 2001, San Jose, CA, pp.130--140, 2001.
• 2: Alan Hanjalic , Li-Qun Xu, User-oriented Affective Video Content Analysis, Proceedings of the IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL'01), p.50, December 14-14, 2001
• 3: Simon Moncrieff , Chitra Dorai , Svetha Venkatesh, Affect computing in film through sound energy dynamics, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500231]
• 4: Lang, P.: The emotion probe: Studies of motivation and attention, American Psychologist, 50(5), pp. 372--385, 1995.
• 5: Rosalind W. Picard, Affective computing, MIT Press, Cambridge, MA, 1997
• 6: Valdez, P. and Mehrabian, A.: Effects of color on emotions, Journal of Experimental Psychology: General, pp. 394--409, 1994.
• 7: Scheirer, J. and Picard, R.: Affective Objects, MIT Media lab Technical Rep. No 524.
• 8: Goldstein, E. : Sensation and Perception, Brooks/Cole, 1999.
• 9: Lee, S., and Hayes, M.: Real-time camera motion classification for content-based indexing and retrieval using templates, Proc. ICASSP, pp.3664--3667, 2002.
• 10: Lawrence Rabiner , Biing-Hwang Juang, Fundamentals of speech recognition, Prentice-Hall, Inc., Upper Saddle River, NJ, 1993
• 11: Boreczky, J. and Wilcox, E.: A Hidden Markov Model Framework for Video Segmentation Using Audio and Image Features, Proc. ICASSP' 98 , 1998.
• 12: Eickeler, S. and Muller, S.: Content-based Video Indexing of TV Broadcast News Using Hidden Markov Models, Proc. ICASSP'99 , 1999.
• 13: Baoxin Li , M. Ibrahim Sezan, Event Detection and Summarization in Sports Video, Proceedings of the IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL'01), p.132, December 14-14, 2001
• 14: Milind R. Naphade , Ashutosh Garg , Thomas S. Huang, Audio-Visual Event Detection using Duration dependent input output Markov models, Proceedings of the IEEE Workshop on Content-based Access of Image and Video Libraries (CBAIVL'01), p.30, December 14-14, 2001
• 15: Zhang, H., Wu, J., Zhong, D., and Smoliar, S.: An integrated system for content-based video retrieval and browsing, " Pattern Recognition, Vol. 30, pp.643--58, 1997.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms

Keywords:
content analysis, emotional event, hidden Markov models