Improving SVM accuracy by training on auxiliary data sources

ABSTRACT

The standard model of supervised learning assumes that training and test data are drawn from the same underlying distribution. This paper explores an application in which a second, auxiliary, source of data is available drawn from a different distribution. This auxiliary data is more plentiful, but of significantly lower quality, than the training and test data. In the SVM framework, a training example has two roles: (a) as a data point to constrain the learning process and (b) as a candidate support vector that can form part of the definition of the classifier. The paper considers using the auxiliary data in either (or both) of these roles. This auxiliary data framework is applied to a problem of classifying images of leaves of maple and oak trees using a kernel derived from the shapes of the leaves. Experiments show that when the training data set is very small, training with auxiliary data can produce large improvements in accuracy, even when the auxiliary data is significantly different from the training (and test) data. The paper also introduces techniques for adjusting the kernel scores of the auxiliary data points to make them more comparable to the training data points.

REFERENCES

• 1: Claus Bahlmann , Bernard Haasdonk , Hans Burkhardt, On-Line Handwriting Recognition with Support Vector Machines " A Kernel Approach, Proceedings of the Eighth International Workshop on Frontiers in Handwriting Recognition (IWFHR'02), p.49, August 06-08, 2002
• 2: Kristin P. Bennett , Ayhan Demiriz, Semi-supervised support vector machines, Proceedings of the 1998 conference on Advances in neural information processing systems II, p.368-374, July 1999
• 3: Avrim Blum , Tom Mitchell, Combining labeled and unlabeled data with co-training, Proceedings of the eleventh annual conference on Computational learning theory, p.92-100, July 24-26, 1998, Madison, Wisconsin, United States [doi>10.1145/279943.279962]
• 4: Clark, P., & Matwin, S. (1993). Using qualitative models to guide inductive learning. Machine Learning: Proceedings of the Tenth International Conference (pp. 49--56). San Francisco, CA: Morgan Kaufmann.
• 5: Nello Cristianini , John Shawe-Taylor, An introduction to support Vector Machines: and other kernel-based learning methods, Cambridge University Press, New York, NY, 1999
• 6: Durbin, R., Eddy, S., Krogh, A., & Mitchison, G. (1998). Biological sequence analysis: Probabilistic models of proteins and nucleic acids. Cambridge: Cambridge University Press.
• 7: Graepel, T., Herbrich, R., Scholkopf, B., Smola, A., Bartlett, P., Robert-Muller, K., Obermayer, K., & Williamson, B. (1999). Classification on proximity data with LP---machines. Proceedings of the Ninth International Conference on Artificial Neural Networks (pp. 304--309).
• 8: Mangasarian, O. (2000). Generalized support vector machines. In A. J. Smola, P. L. Bartlett, B. Schlkopf and D. Schuurmans (Eds.), Advances in large margin classifiers, 135--146. Cambridge, MA.: MIT Press.
• 9: Milios, E., & Petrakis, E. (2000). Shape retrieval based on dynamic programming. IEEE Transactions on Image Processing, 8, 141--146.
• 10: Euripides G. M. Petrakis , Aristeidis Diplaros , Evangelos Milios, Matching and Retrieval of Distorted and Occluded Shapes Using Dynamic Programming, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.11, p.1501-1516, November 2002 [doi>10.1109/TPAMI.2002.1046166]