VideoMule: a consensus learning approach to multi-label classification from noisy user-generated videos

ABSTRACT

With the growing proliferation of conversational media and devices for generating multimedia content, the Internet has seen an expansion in websites catering to user-generated media. Most of the user-generated content is multimodal in nature as it has videos, audio, text (in the form of tags), comments and so on. Content analysis is a challenging problem on this type of media since it is noisy, unstructured and unreliable. In this paper we propose VideoMule, a consensus learning approach for multi-label video classification from noisy user-generated videos. In our scheme, we train classification and clustering algorithms on individual modes of information such as user comments, tags, video features and so on. We then combine the results of trained classifiers and clustering algorithms using a novel heuristic consensus learning algorithm which as a whole performs better than each individual learning model.

REFERENCES

• 1: S. Baluja, R. Seth, D. Sivakumar, Y. Jing, J. Yagnik, S. Kumar, D. Ravichandran, and M. Aly. Video suggestion and discovery for youtube: taking random walks through the view graph. In WWW, pages 895--904, 2008.
• 2: C. M. Bishop. Chapter 8: Graphical models. In Pattern Recognition and Machine Learning. Springer., pages 359--418, 2006.
• 3: X. Cheng, C. Dale, and J. Liu. Understanding the characteristics of internet short video sharing: Youtube as a case study. In CoRR abs, 2007.
• 4: K. Ganchev, J. Graca, J. Blitzer, and B. Taskar. Multi-view learning over structured and non-identical outputs. In UAI, pages 204--211, 2008.
• 5: J. Gao, W. Fan, Y. Sun, and J. Han. Heterogeneous source consensus learning via decision propagation and negotiation. In KDD, pages 339--348, 2009.
• 6: J. Han and M. Kamber. Data mining: Concepts and techniques. In Morgan Kauffman, 2000.
• 7: A. Klaser, M. Marszalek, and C. Schmid. A spatio-temporal descriptor based on 3d-gradients. In British Machine Vision Conference 2008.
• 8: L. Liu, Y. Rui, L.-F. Sun, B. Yang, J. Zhang, and S.-Q. Yang. Topic mining on web-shared videos. In ICASSP, pages 2145--2148, 2008.
• 9: L. Liu, L. Sun, Y. Rui, Y. Shi, and S. Yang. Web video topic discovery and tracking via bipartite graph reinforcement model. In WWW, pages 1009--1018, 2008.
• 10: D. Zhou, O. Bousquet, T. N. Lal, J. Weston, and B. Scholkopf. Learning with local and global consistency. In NIPS, pages 321--328, 2004.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Video (e.g., tape, disk, DVI)

General Terms:
Algorithms

Keywords:
multimodal information processing, video classification