Gaining insights into support vector machine pattern classifiers using projection-based tour methods

ABSTRACT

This paper discusses visual methods that can be used to understand and interpret the results of classification using support vector machines (SVM) on data with continuous real-valued variables. SVM induction algorithms build pattern classifiers by identifying a maximal margin separating hyperplane from training examples in high dimensional pattern spaces or spaces induced by suitable nonlinear kernel transformations over pattern spaces. SVM have been demonstrated to be quite effective in a number of practical pattern classification tasks. Since the separating hyperplane is defined in terms of more than two variables it is necessary to use visual techniques that can navigate the viewer through high-dimensional spaces. We demonstrate the use of projection-based tour methods to gain useful insights into SVM classifiers with linear kernels on 8-dimensional data.

REFERENCES

• 1: Daniel Asimov, The grand tour: a tool for viewing multidimensional data, SIAM Journal on Scientific and Statistical Computing, v.6 n.1, p.128-143, Jan. 1985 [doi>10.1137/0906011]
• 2: M. Brown, W. Grundy, D. Lin, N. Christianini, C. Sugnet, T. Furey, M. Ares Jr., and D. Haussler. Knowledge based analysis of microarray gene expression data using support vector machines. TR UCSC CRL-99-09, CRL, Santa Cruz, CA., 1999.
• 3: A. Buja, D. Asimov, C. Hurley, and J. A. McDonald. Elements of a Viewing Pipeline for Data Analysis. In W. S. Cleveland and M. E. McGill editors, Dynamic Graphics for Statistics, pages 277-308. Wadsworth, Monterey, CA, 1988.
• 4: A. Buja, D. Cook, D. Asimov, and C. Hurley. Dynamic Projections in High-Dimensional Visualization: Theory and Computational Methods. TR, AT&T Labs, Florham Park, N J, 1997.
• 5: D. Caragea, A. Silvescu, and V. Honavar. Agents that learn from distributed dynamic data sources. In Proc. of the Workshop on Learning Agents, Agents 2000/ECML 2000, Barcelona, Spain, 53-61, 2000.
• 6: D. Caragea, A. Silvescu, and V. Honavar. Analysis and synthesis of agents that learn from distributed dynamic data sources. In S. E. A. Wermter, Ed., Neural Network Architectures Inspired by Neuroscience. Springer-Verlag, 2001.
• 7: D. Cook and A. Buja. Manual Controls For High-Dimensional Data Projections. Journal of Computational and Graphical Statistics, 6(4):464-480, 1997. Also see www.public, iastate, edu/ ~dicook/researchlpapers/manip. html.
• 8: D. Cook, A. Buja, J. Cabrera, and C. Hurley. Grand Tour and Projection Pursuit. Journal of Comp. and Graphical Statistics, 4(3):155-172, 1995.
• 9: Corinna Cortes , Vladimir Vapnik, Support-Vector Networks, Machine Learning, v.20 n.3, p.273-297, Sept. 1995 [doi>10.1023/A:1022627411411]
• 10: D. J. Donnell, A. Buja, and W. Stuetzle. Analysis of Additive Dependencies using Smallest Additive Principle Components. Annals of Statistics, 22, 1994.
• 11: M. Forina, C. Armanino, S. Lanteri, and E. Tiscornia. Classification of olive oils from their fatty acid composition. In H. Martens and H. Russwurm Jr., editors, Food Research and Data Analysis, pages 189-214. Applied Science Publishers, London, 1983.
• 12: G. Furnas and A. Buja. Prosection Views: Dimensional Inference Through Sections and Projections. Journal of Computational and Graphical Statistics, 3(4):323-385, 1994.
• 13: Marti A. Hearst, Support Vector Machines, IEEE Intelligent Systems, v.13 n.4, p.18-28, July 1998 [doi>10.1109/5254.708428]
• 14: T. Joachims. Making Large-Scale SVM Learning Practical. MIT Press, 1999.
• 15: Vladimir N. Vapnik, The nature of statistical learning theory, Springer-Verlag New York, Inc., New York, NY, 1995
• 16: wgg. public, iastate, edu/~dicook/JSS/paper
• 17: Limn: www.public.iastate.edu/ dicook/Limn

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Classifier design and evaluation

Additional Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Pattern analysis
  I.6 SIMULATION AND MODELING
      I.6.8 Types of Simulation
          Subjects: Visual

General Terms:
Algorithms, Design, Performance

Keywords:
Dynamic graphics, classification, machine leaning, multivariate data, support vector machines, tours, visualization