Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

The Support Vector Machine (SVM) is an acknowledged powerful tool for building classifiers, but it lacks flexibility, in the sense that the kernel is chosen prior to learning. Multiple Kernel Learning (MKL) enables to learn the kernel, from an ensemble of basis kernels, whose combination is optimized in the learning process. Here, we propose Composite Kernel Learning to address the situation where distinct components give rise to a group structure among kernels. Our formulation of the learning problem encompasses several setups, putting more or less emphasis on the group structure. We characterize the convexity of the learning problem, and provide a general wrapper algorithm for computing solutions. Finally, we illustrate the behavior of our method on multi-channel data where groups correpond to channels.

REFERENCES

• 1: Francis R. Bach , Gert R. G. Lanckriet , Michael I. Jordan, Multiple kernel learning, conic duality, and the SMO algorithm, Proceedings of the twenty-first international conference on Machine learning, p.6, July 04-08, 2004, Banff, Alberta, Canada [doi>10.1145/1015330.1015424]
• 2: Blankertz, B., Müller, K.-R., Curio, G., Vaughan, T. M., Schalk, G., Wolpaw, J. R., Schlögl, A., Neuper, C., Pfurtscheller, G., Hinterberger, T., Schröder, M., & Birbaumer, N. (2004). The BCI competition 2003: progress and perspectives in detection and discrimination of EEG single trials. IEEE Trans. Biomed. Eng, 51, 1044--1051.
• 3: J. Frédéric Bonnans , Alexander Shapiro, Optimization Problems with Perturbations: A Guided Tour, SIAM Review, v.40 n.2, p.228-264, June 1998 [doi>10.1137/S0036144596302644]
• 4: Stephen Boyd , Lieven Vandenberghe, Convex Optimization, Cambridge University Press, New York, NY, 2004
• 5: Olivier Chapelle , Vladimir Vapnik , Olivier Bousquet , Sayan Mukherjee, Choosing Multiple Parameters for Support Vector Machines, Machine Learning, v.46 n.1-3, p.131-159, 2002 [doi>10.1023/A:1012450327387]
• 6: Nello Cristianini , Colin Campbell , John Shawe-Taylor, Dynamically adapting kernels in support vector machines, Proceedings of the 1998 conference on Advances in neural information processing systems II, p.204-210, July 1999
• 7: Cristianini, N., Shawe-Taylor, J., Elisseeff, A., & Kandola, K. (2002). On kernel-target alignment. Advances in Neural Information Processing Systems 14 (pp. 367--373). MIT Press.
• 8: Grandvalet, Y., & Canu, S. (2003). Adaptive scaling for feature selection in SVMs. Advances in Neural Information Processing Systems 15 (pp. 569--576). MIT Press.
• 9: Isabelle Guyon , André Elisseeff, An introduction to variable and feature selection, The Journal of Machine Learning Research, 3, 3/1/2003
• 10: Gert R. G. Lanckriet , Nello Cristianini , Peter Bartlett , Laurent El Ghaoui , Michael I. Jordan, Learning the Kernel Matrix with Semidefinite Programming, The Journal of Machine Learning Research, 5, p.27-72, 12/1/2004
• 11: Alain Rakotomamonjy , Francis Bach , Stéphane Canu , Yves Grandvalet, More efficiency in multiple kernel learning, Proceedings of the 24th international conference on Machine learning, p.775-782, June 20-24, 2007, Corvalis, Oregon [doi>10.1145/1273496.1273594]
• 12: Rakotomamonjy, A., Guigue, V., Mallet, G., & Alvarado, V. (2005). Ensemble of SVMs for improving brain-computer interface P300 speller performances. 15th International Conference on Artificial Neural Networks (pp. 45--50). Springer.
• 13: Bernhard Scholkopf , Alexander J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, MIT Press, Cambridge, MA, 2001
• 14: Michael Schröder , Thomas Navin Lal , Thilo Hinterberger , Martin Bogdan , N. Jeremy Hill , Niels Birbaumer , Wolfgang Rosenstiel , Bernhard Schölkopf, Robust EEG channel selection across subjects for brain-computer interfaces, EURASIP Journal on Applied Signal Processing, v.2005 n.1, p.3103-3112, 1 January 2005 [doi>10.1155/ASP.2005.3103]
• 15: Sören Sonnenburg , Gunnar Rätsch , Christin Schäfer , Bernhard Schölkopf, Large Scale Multiple Kernel Learning, The Journal of Machine Learning Research, 7, p.1531-1565, 12/1/2006
• 16: Szafranski, M., Grandvalet, Y., & Morizet-Mahoudeaux, P. (2008). Hierarchical penalization. In J. Platt, D. Koller, Y. Singer and S. Roweis (Eds.), Advances in neural information processing systems 20. MIT Press.
• 17: Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B, 58, 267--288.
• 18: Weston, J., Mukherjee, S., Chapelle, O., Pontil, M., Poggio, T., & Vapnik, V. (2001). Feature selection for SVMs. Advances in Neural Information Processing Systems 13 (pp. 668--674). MIT Press.
• 19: Yuan, M., & Lin, Y. (2006). Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society. Series B, 68, 49--67.
• 20: Zhao, P., Rocha, G., & Yu, B. (to appear). The composite absolute penalties family for grouped and hierarchical variable selection. Annals of Statistics.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning

Additional Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Classifier design and evaluation
  I.6 SIMULATION AND MODELING
      I.6.4 Model Validation and Analysis
      I.6.5 Model Development
          Subjects: Modeling methodologies

General Terms:
Algorithms, Design, Performance