Collaborative learning for image and video annotation

ABSTRACT

Classical machine learning methods, such as Support Vector Machines, by taking each concept detection as an independent classification problem, can not achieve a sound performance for image and video annotation due to the overfitting problems. Thus, some prior knowledge is required to assist the learning of independent concept detectors, e.g. some concepts look much more alike to each other. In this paper, we assume that visually similar concepts should share resembled detectors. Based on the assumption, Collaborative Learning is proposed, to incorporate cross-concept collaborations into the joint learning of similar detectors over related concepts. Besides the collaborations, different concepts should also perform discriminations for classifying each other. To benefit from different trade-offs between collaboration and discrimination, we propose Multi-Granularity Boosting strategy, where each granularity corresponds to a specific balance between collaboration and discrimination for Collaborative Learning. The ultimate concept detector is an additive model that combines classifiers under different collaboration granularities together. Evaluations on both image and video annotation benchmark demonstrate that our method achieves a superior performance over independent annotation.

REFERENCES

• 1: Trecvid. http://www-nlpir.nist.gov/projects/trecvid/.
• 2: A. Amir and et al. Ibm research trecvid-2003 video retrieval system. In TRECVID Proceedings, 2003.
• 3: N. Aronszajn. Theory of reproducing kernels. Trans. Am. Math. Soc., (68):337--404, 1950.
• 4: Evgeniy Bart , Shimon Ullman, Cross-Generalization: Learning Novel Classes from a Single Example by Feature Replacement, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1, p.672-679, June 20-26, 2005 [doi>10.1109/CVPR.2005.117]
• 5: Stephen Boyd , Lieven Vandenberghe, Convex Optimization, Cambridge University Press, New York, NY, 2004
• 6: S.-F. Chang and et al. Columbia university trecvid-2006 video search and high-level feature extraction. In TRECVID Proceedings, 2006.
• 7: Jianping Fan , Yuli Gao , Hangzai Luo, Hierarchical classification for automatic image annotation, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277763]
• 8: J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: a statistical view of boosting. In Annals of statistics, 2000.
• 9: A. Hauptmann, M.-Y. Chen, and M. Christel. Confounded expectations: Informedia at trecvid 2004. In TRECVID Proceedings, 2004.
• 10: George A. Miller, WordNet: a lexical database for English, Communications of the ACM, v.38 n.11, p.39-41, Nov. 1995 [doi>10.1145/219717.219748]
• 11: P. Moreno, P. Ho, and N. Vasconcelos. A kullback-leibler divergence based kernel for svm classification in multimedia applications. In NIPS, 2003.
• 12: M. R. Naphade. Statistical techniques in video data management. In IEEE Workshop on Multimedia Signal Processing, 2002.
• 13: M. R. Naphade and et al. A light scale concept ontology for multimedia. Technical report, 2005.
• 14: M. R. Naphade, I. Kozintsev, and T. S. Huang. Factor graph framework for semantic video indexing. IEEE Trans. on CSVT, 12(1), Jan. 2002.
• 15: Guo-Jun Qi , Xian-Sheng Hua , Yong Rui , Jinhui Tang , Tao Mei , Hong-Jiang Zhang, Correlative multi-label video annotation, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291245]
• 16: Antonio Torralba , Kevin P. Murphy , William T. Freeman, Sharing Visual Features for Multiclass and Multiview Object Detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.29 n.5, p.854-869, May 2007 [doi>10.1109/TPAMI.2007.1055]
• 17: V. N. Vapnik. Statistical Learning Theory.Wiley-Interscience, 1998.
• 18: Lei Wu , Xian-Sheng Hua , Nenghai Yu , Wei-Ying Ma , Shipeng Li, Flickr distance, Proceeding of the 16th ACM international conference on Multimedia, October 26-31, 2008, Vancouver, British Columbia, Canada [doi>10.1145/1459359.1459364]
• 19: Rong Yan , Jelena Tesic , John R. Smith, Model-shared subspace boosting for multi-label classification, Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, August 12-15, 2007, San Jose, California, USA [doi>10.1145/1281192.1281281]
• 20: A. Yanagawa, S.-F. Chang, L. Kennedy, andW. Hsu. Columbia university's baseline detectors for 374 lscom semantic visual concepts. Columbia University ADVENT Technical Report, (222--2006--8), 2007.
• 21: Jun Yang , Rong Yan , Alexander G. Hauptmann, Cross-domain video concept detection using adaptive svms, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291276]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Video analysis

General Terms:
Algorithms, Experimentation, Theory

Keywords:
concept correlation, image annotation, video annotation