Domain adaptation from multiple sources via auxiliary classifiers

ABSTRACT

We propose a multiple source domain adaptation method, referred to as Domain Adaptation Machine (DAM), to learn a robust decision function (referred to as target classifier) for label prediction of patterns from the target domain by leveraging a set of pre-computed classifiers (referred to as auxiliary/source classifiers) independently learned with the labeled patterns from multiple source domains. We introduce a new data-dependent regularizer based on smoothness assumption into Least-Squares SVM (LS-SVM), which enforces that the target classifier shares similar decision values with the auxiliary classifiers from relevant source domains on the unlabeled patterns of the target domain. In addition, we employ a sparsity regularizer to learn a sparse target classifier. Comprehensive experiments on the challenging TRECVID 2005 corpus demonstrate that DAM outperforms the existing multiple source domain adaptation methods for video concept detection in terms of effectiveness and efficiency.

REFERENCES

• 1: Mikhail Belkin , Partha Niyogi , Vikas Sindhwani, Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples, The Journal of Machine Learning Research, 7, p.2399-2434, 12/1/2006
• 2: Blitzer, J., McDonald, R., & Pereira, F. (2006). Domain adaptation with structural correspondence learning. Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, 120--128.
• 3: Borgwardt, K. M., Gretton, A., Rasch, M. J., Kriegel, H.-P., Schölkopf, B., & Smola, A. J. (2006). Integrating structured biological data by kernel maximum mean discrepancy. Proceedings of the 14th International Conference on Intelligent Systems for Molecular Biology, 49--57.
• 4: Koby Crammer , Michael Kearns , Jennifer Wortman, Learning from Multiple Sources, The Journal of Machine Learning Research, 9, p.1757-1774, 6/1/2008
• 5: Daumé III, H. (2007). Frustratingly easy domain adaptation. Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, 256--263.
• 6: Theodoros Evgeniou , Charles A. Micchelli , Massimiliano Pontil, Learning Multiple Tasks with Kernel Methods, The Journal of Machine Learning Research, 6, p.615-637, 12/1/2005
• 7: Huang, J., Smola, A., Gretton, A., Borgwardt, K. M., & Schöölkopf, B. (2007). Correcting sample selection bias by unlabeled data. Advances in Neural Information Processing Systems 19, 601--608.
• 8: Thorsten Joachims, Transductive Inference for Text Classification using Support Vector Machines, Proceedings of the Sixteenth International Conference on Machine Learning, p.200-209, June 27-30, 1999
• 9: Kato, T., Kashima, H., Sugiyama, M., & Asai, K. (2008). Multi-task learning via conic programming. Advances in Neural Information Processing Systems 20, 737--744.
• 10: Ping Luo , Fuzhen Zhuang , Hui Xiong , Yuhong Xiong , Qing He, Transfer learning from multiple source domains via consensus regularization, Proceeding of the 17th ACM conference on Information and knowledge management, October 26-30, 2008, Napa Valley, California, USA [doi>10.1145/1458082.1458099]
• 11: Mansour, Y., Mohri, M., & Rostamizadeh, A. (2009). Domain adaptation with multiple sources. Advances in Neural Information Processing Systems 21, 1041--1048.
• 12: Naphade, M. R., Kennedy, L., Kender, J. R., Chang, S.-F., Smith, J. R., Over, P., & Hauptmann, A. (2005). A light scale concept ontology for multimedia understanding for TRECVID 2005 (Technical Report). IBM Research Technical Report.
• 13: Rosenstein, M. T., Marx, Z., & Kaelbling, L. P. (2005). To transfer or not to transfer. NIPS 2005 Workshop on Inductive Transfer: 10 Years Later.
• 14: Schweikert, G., Widmer, C., Schölkopf, B., & Räätsch, G. (2009). An empirical analysis of domain adaptation algorithm for genomic sequence analysis. Advances in Neural Information Processing Systems 21, 1433--1440.
• 15: Vikas Sindhwani , Partha Niyogi , Mikhail Belkin, Beyond the point cloud: from transductive to semi-supervised learning, Proceedings of the 22nd international conference on Machine learning, p.824-831, August 07-11, 2005, Bonn, Germany [doi>10.1145/1102351.1102455]
• 16: Alan F. Smeaton , Paul Over , Wessel Kraaij, Evaluation campaigns and TRECVid, Proceedings of the 8th ACM international workshop on Multimedia information retrieval, October 26-27, 2006, Santa Barbara, California, USA [doi>10.1145/1178677.1178722]
• 17: Storkey, A., & Sugiyama, M. (2007). Mixture regression for covariate shift. Advances in Neural Information Processing Systems 19, 1337--1344.
• 18: Tony Van Gestel , Johan A. K. Suykens , Bart Baesens , Stijn Viaene , Jan Vanthienen , Guido Dedene , Bart De Moor , Joos Vandewalle, Benchmarking Least Squares Support Vector Machine Classifiers, Machine Learning, v.54 n.1, p.5-32, January 2004 [doi>10.1023/B:MACH.0000008082.80494.e0]
• 19: Pengcheng Wu , Thomas G. Dietterich, Improving SVM accuracy by training on auxiliary data sources, Proceedings of the twenty-first international conference on Machine learning, p.110, July 04-08, 2004, Banff, Alberta, Canada [doi>10.1145/1015330.1015436]
• 20: Jun Yang , Rong Yan , Alexander G. Hauptmann, Cross-domain video concept detection using adaptive svms, Proceedings of the 15th international conference on Multimedia, September 25-29, 2007, Augsburg, Germany [doi>10.1145/1291233.1291276]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Classifier design and evaluation

Additional Classification:
  G. Mathematics of Computing
  G.1 NUMERICAL ANALYSIS
      G.1.6 Optimization
          Subjects: Least squares methods

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Pattern analysis
  I.6 SIMULATION AND MODELING
      I.6.4 Model Validation and Analysis
      I.6.5 Model Development
          Subjects: Modeling methodologies

General Terms:
Algorithms, Design, Performance, Theory