Affect computing in film through sound energy dynamics

ABSTRACT

We develop an algorithm for the detection and classification of affective sound events underscored by specific patterns of sound energy dynamics. We relate the portrayal of these events to proposed high level affect or emotional coloring of the events. In this paper, four possible characteristic sound energy events are identified that convey well established meanings through their dynamics to portray and deliver certain affect, sentiment related to the horror film genre. Our algorithm is developed with the ultimate aim of automatically structuring sections of films that contain distinct shades of emotion related to horror themes for nonlinear media access and navigation. An average of 82% of the energy events, obtained from the analysis of the audio tracks of sections of four sample films corresponded correctly to the proposed affect. While the discrimination between certain sound energy event types was low, the algotithm correctly detected 71% of the occurrences of the sound energy events within audio tracks of the films analyzed, and thus forms a useful basis for determining affective scenes characteristic of horror in movies.

REFERENCES

• 1: R. Deriche. Recursively implementing the Gaussian and it's derivatives. In ICIP'92, Proc. 2nd Singapore Int. Conf on Image Processing, pages 263-267, 1992.
• 2: M. Huckvale. Speech filing system. Url: www.phon.ucl.ac.uk/resources/sfs/.
• 3: S. Moncrieff, C. Dorai, and S. Venkatesh. Detecting indexical signs in film audio for scene interpretation. To appear, ICME 2001.
• 4: Silvia Pfeiffer , Stephan Fischer , Wolfgang Effelsberg, Automatic Audio Content Analysis, University of Mannheim, 1996
• 5: Hari Sundaram , Shih-Fu Chang, Determining computable scenes in films and their structures using audio-visual memory models, Proceedings of the eighth ACM international conference on Multimedia, p.95-104, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.354440]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing