Analyzing discussion scene contents in instructional videos

ABSTRACT

This paper describes our current effort on analyzing the contents of discussion scenes in instructional videos based on a clustering technique. Specifically, given a discussion scene pre-detected from an education or training video, we first apply a mode-based clustering approach to group all speech segments into an optimal number of clusters where each cluster contains speech from one speaker; we then analyze the discussion patterns in the scene, and subsequently classify it into either a 2-speaker or multi-speaker discussion. Encouraging classification results have been achieved on 122 discussion scenes detected from five IBM MicroMBA videos. Moreover, we have also observed fairly good performance on the speaker clustering scheme, which demonstrates the superiority of the proposed clustering approach. Undoubtedly, the discussion scene information output from this analysis scheme would facilitate the content browsing, searching and understanding of instructional videos.

REFERENCES

• 1: S. Chen and P. S. Gopalakrishnan, "Speaker, environment and channel change detection and clustering via the Bayesian information criterion," Proc. of DARPA Broadcast News Transcription and Understanding Workshop, 1998.
• 2: H. Jin, F. Kubala, and R. Schwartz, "Automatic speaker clustering," Proc. of the Speech Recognition Workshop, Chantilly, February 1997.
• 3: M. Siegler and et al., "Automatic segmentation, classification, and clustering of broadcast news," Proc. of Speech Recognition Workshop, 1997.
• 4: A. Divakaran, Video Summarization Using MPEG-7 Motion Activity and Audio Descriptors. Video Mining, Kluwer Academic Publishers, 2003.
• 5: Y. Li and C. Dorai, "SVM-based audio classification for instructional video analysis," ICASSP'04, 2004.
• 6: F. Zheng and et al., "The distance measure for line spectrum pairs applied to speech recognition," ICSLP'98, 1998.
• 7: Y. Li and C. Dorai, "Detecting discussion scenes in instructional video," ICME'04, 2004.
• 8: Anil K. Jain , Richard C. Dubes, Algorithms for clustering data, Prentice-Hall, Inc., Upper Saddle River, NJ, 1988
• 9: C. Sugar and G. James, "Finding the number of clusters in a data set: An information theoretic approach," Journal of the American Statistical Association, pp. 750--763, 1998.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Clustering

General Terms:
Algorithms, Design, Experimentation

Keywords:
discussion pattern detection, discussion scene, e-learning, instructional video content analysis, speaker clustering