Joke-o-mat: browsing sitcoms punchline by punchline

ABSTRACT

This paper summarizes our contribution to the Yahoo! task of the ACM Multimedia Grand Challenge. This challenge asks for the robust automatic segmentation of videos according to "narrative themes". Based on the automatic segmentation methods presented in [1] and partly [2], we describe a system to navigate Seinfeld episodes based on automatic segmentation of the audio track only. The system distinguishes laughter, music, and other noise as well as speech segments. Speech segments are identified against pre-trained speaker models of the actors. Given this segmentation and the artistic production rules that underlie the genre situation comedy and Seinfeld in particular, the system enables a user to browse an episode by scene, by punchline, and by dialog segments. The themes can be filtered by the main actors, e.g. the user can select to see only punchlines by Jerry and Kramer. Based on the length of the laughter, the top 5 punchlines are also identified and presented to the user.

REFERENCES

• 1: G. Friedland and O. Vinyals. Live speaker identification in conversations. In Proceedings of ACM Multimedia, pages 1017--1018. ACM, October 2008.
• 2: G. Friedland, C. Yeo, and H. Hung. Visual speaker localization aided by acoustic models. In Proceedings of ACM Multimedia, to appear. ACM, October 2009.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.5 Sound and Music Computing
          Subjects: Signal analysis, synthesis, and processing

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.4 Hypertext/Hypermedia
          Subjects: Navigation
      H.5.5 Sound and Music Computing
          Subjects: Systems

General Terms:
Algorithms, Human Factors

Keywords:
acoustic event detection, speaker ID, video navigation