Nomograms for visualizing support vector machines

ABSTRACT

We propose a simple yet potentially very effective way of visualizing trained support vector machines. Nomograms are an established model visualization technique that can graphically encode the complete model on a single page. The dimensionality of the visualization does not depend on the number of attributes, but merely on the properties of the kernel. To represent the effect of each predictive feature on the log odds ratio scale as required for the nomograms, we employ logistic regression to convert the distance from the separating hyperplane into a probability. Case studies on selected data sets show that for a technique thought to be a black-box, nomograms can clearly expose its internal structure. By providing an easy-to-interpret visualization the analysts can gain insight and study the effects of predictive factors.

REFERENCES

• 1: Yasemin Altun , Alex J. Smola , Thomas Hofmann, Exponential families for conditional random fields, Proceedings of the 20th conference on Uncertainty in artificial intelligence, p.2-9, July 07-11, 2004, Banff, Canada
• 2: G. W. Brier. Verification of forecasts expressed in terms of probability. Weather Rev, 78:1--3, 1950.
• 3: C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines, 2005. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
• 4: J. Demšar and B. Zupan. Orange: From experimental machine learning to interactive data mining, 2004. White Paper (http://www.ailab.si/orange) Faculty of Computer and Information Science, University of Ljubljana, Slovenia.
• 5: S. R. Gunn , J. S. Kandola, Structural Modelling with Sparse Kernels, Machine Learning, v.48 n.1-3, p.137-163, 2002 [doi>10.1023/A:1013903804720]
• 6: T. L. Hankins. Blood, dirt, and nomograms: A particular history of graphs. ISIS: Journal of the History of Science in Society, 90:50--80, 1999.
• 7: F. E. Harrell. Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis. Springer, New York, 2001.
• 8: D. Harrison and D. L. Rubinfeld. Hedonic prices and the demand for clean air. J Environ Economics & Management, 5:81--102, 1978.
• 9: T. Hastie and R. Tibshirani. Generalized Additive Models. Chapman and Hall, London, 1990.
• 10: S. Hettich and S. D. Bay. The UCI KDD archive. Irvine, CA: University of California, Department of Information and Computer Science, 1999.
• 11: D. W. Hosmer and S. Lemeshow. Applied Logistic Regression. John Wiley & Sons, New York, 2000.
• 12: A. Jakulin. Extensions to the Orange data mining framework, January 2002. http://www.ailab.si/aleks/orng/.
• 13: A. Jakulin and I. Bratko. Analyzing attribute dependencies. In N. Lavrač, D. Gamberger, H. Blockeel, and L. Todorovski, editors, Proc. of Principles of Knowledge Discovery in Data (PKDD), volume 2838 of LNAI, pages 229--240. Springer-Verlag, September 2003.
• 14: Aleks Jakulin , Ivan Bratko, Testing the significance of attribute interactions, Proceedings of the twenty-first international conference on Machine learning, p.52, July 04-08, 2004, Banff, Alberta, Canada [doi>10.1145/1015330.1015377]
• 15: M. W. Kattan, J. A. Eastham, A. M. Stapleton, T. M. Wheeler, and P. T. Scardino. A preoperative nomogram for disease recurrence following radical prostatectomy for prostate cancer. J Natl Cancer Inst, 90(10):766--71, 1998.
• 16: John Lafferty , Xiaojin Zhu , Yan Liu, Kernel conditional random fields: representation and clique selection, Proceedings of the twenty-first international conference on Machine learning, p.64, July 04-08, 2004, Banff, Alberta, Canada [doi>10.1145/1015330.1015337]
• 17: J. Lubsen, J. Pool, and E. van der Does. A practical device for the application of a diagnostic or prognostic function. Methods of Information in Medicine, 17:127--129, 1978.
• 18: A. J. Miller. Algorithm AS 274: Least squares routines to supplement those of Gentleman. Appl. Statist., 41(2):458--478, 1992.
• 19: Martin Možina , Janez Demšar , Michael Kattan , Blaž Zupan, Nomograms for visualization of naive Bayesian classifier, Proceedings of the 8th European Conference on Principles and Practice of Knowledge Discovery in Databases, p.337-348, September 20-24, 2004, Pisa, Italy
• 20: K. Pelckmans, J.A.K. Suykens, and B. De Moor. Building sparse representations and structure determination on LS-SVM substrates. Neurocomputing, 64:137--159, March 2005.
• 21: J. C. Platt. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In Advances in Large Margin Classifiers. MIT Press, 1999.
• 22: B. Schölkopf and A. J. Smola. Learning with Kernels. MIT Press, 2002.
• 23: Vladimir N. Vapnik, The nature of statistical learning theory, Springer-Verlag New York, Inc., New York, NY, 1995

INDEX TERMS

Primary Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Multivariate statistics

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Theory and methods

General Terms:
Human Factors, Theory

Keywords:
machine learning, nomogram, support vector machines, visualization