News video classification using SVM-based multimodal classifiers and combination strategies

ABSTRACT

Video classification is the first step toward multimedia content understanding. When video is classified into conceptual categories, it is usually desirable to combine evidence from multiple modalities. However, combination strategies in previous studies were usually ad hoc. We investigate a meta-classification combination strategy using Support Vector Machine, and compare it with probability-based strategies. Text features from closed-captions and visual features from images are combined to classify broadcast news video. The experimental results show that combining multimodal classifiers can significantly improve recall and precision, and our meta-classification strategy gives better precision than the approach of taking the product of the posterior probabilities.

REFERENCES

• 1: Ricardo A. Baeza-Yates , Berthier Ribeiro-Neto, Modern Information Retrieval, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1999
• 2: Chang, C.-C. and Lin, C.-J. LIBSVM: a library for support vector machines. 2001. Software available at <u>http://www.csie.ntu.edu.tw/~cjlin/libsvm</u>.
• 3: Nello Cristianini , John Shawe-Taylor, An introduction to support Vector Machines: and other kernel-based learning methods, Cambridge University Press, New York, NY, 1999
• 4: Dimitrova N., Agnihotri, L. and Wei G. Video Classification Based on HMM Using Text and Faces. European Conference on Signal Processing, Finland, 2000.
• 5: Huang, J., Liu, Z., Wang, Y., Chen, Y. and Wong, E.K. Integration of Multimodal Features for Video Scene Classification Based on HMM. IEEE Third Workshop on Multimedia Signal Processing, Copenhagen, Denmark, 1999.
• 6: Thorsten Joachims, Text Categorization with Suport Vector Machines: Learning with Many Relevant Features, Proceedings of the 10th European Conference on Machine Learning, p.137-142, April 21-23, 1998
• 7: Josef Kittler , Mohamad Hatef , Robert P. W. Duin , Jiri Matas, On Combining Classifiers, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.20 n.3, p.226-239, March 1998 [doi>10.1109/34.667881]
• 8: Lin, W.-H., Jin. R. and Hauptmann, A. Meta-classification of Multimedia Classifiers. International Workshop on Knowledge Discovery in Multimedia and Complex Data, Taipei, Taiwan, 2002.
• 9: Liu, Z., Huang, J. and Wang, Y. Classification of TV Programs Based on Audio Information Using Hidden Markov Model. IEEE Workshop on Multimedia Signal Processing, 1998.
• 10: Cheng Lu , Mark S. Drew , James Au, Classification of summarized videos using hidden markov models on compressed chromaticity signatures, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500217]
• 11: Constantine P. Papageorgiou , Michael Oren , Tomaso Poggio, A General Framework for Object Detection, Proceedings of the Sixth International Conference on Computer Vision, p.555, January 04-07, 1998
• 12: Schölkopf, B., Sung, K.-K., Burges, C., Giroso, F., Niyogi, P., Poggio, T. and Vapnik, V. Comparing Support Vector Machine with Gaussian Kernels to Radial Basis Function Classifiers. IEEE Transactions on Signal Processing, 45(11), 1997.
• 13: Vladimir N. Vapnik, The nature of statistical learning theory, Springer-Verlag New York, Inc., New York, NY, 1995
• 14: Howard D. Wactlar , Takeo Kanade , Michael A. Smith , Scott M. Stevens, Intelligent Access to Digital Video: Informedia Project, Computer, v.29 n.5, p.46-52, May 1996 [doi>10.1109/2.493456]
• 15: Wang, Y., Liu, Z. and Huang, J.-C. Multimedia Content Analysis Using Both Audio and Visual Clues. IEEE Signal Processing Magazine, 2000.
• 16: M. F. Porter. An algorithm for suffix stripping. Program, 14(3):130--137, July 1980.