Hyperplane margin classifiers on the multinomial manifold

ABSTRACT

The assumptions behind linear classifiers for categorical data are examined and reformulated in the context of the multinomial manifold, the simplex of multinomial models furnished with the Riemannian structure induced by the Fisher information. This leads to a new view of hyperplane classifiers which, together with a generalized margin concept, shows how to adapt existing margin-based hyperplane models to multinomial geometry. Experiments show the new classification framework to be effective for text classification, where the categorical structure of the data is modeled naturally within the multinomial family.

REFERENCES

• 1: Amari, S., & Nagaoka, H. (2000). Methods of information geometry. American Mathematical Society.
• 2: Bridson, M., & Haefliger, A. (1999). Metric spaces of non-positive curvature, vol. 319 of A Series in Comprehensive Studies in Mathematics. Springer.
• 3: Čencov, N. N. (1982). Statistical decision rules and optimal inference. American Mathematical Society.
• 4: Gous, A. (1998). Exponential and spherical subfamily models. Doctoral dissertation, Stanford University.
• 5: Keith Hall , Thomas Hofmann, Learning Curved Multinomial Subfamilies for Natural Language Processing and Information Retrieval, Proceedings of the Seventeenth International Conference on Machine Learning, p.351-358, June 29-July 02, 2000
• 6: Joachims, T. (2000). The maximum margin approach to learning text classifiers methods, theory and algorithms. Doctoral dissertation, Dortmund University.
• 7: Kass, R. E. (1989). The geometry of asymptotic inference. Statistical Science, 4, 188--234.
• 8: Kass, R. E., & Voss, P. W. (1997). Geometrical foundations of asymptotic inference. John Wiley & Sons, Inc.
• 9: Lafferty, J., & Lebanon, G. (2003). Information diffusion kernels. Advances in Neural Information Processing, 15.
• 10: Lee, J. L. (1997). Riemannian manifolds, an introduction to curvature. Springer.
• 11: Spivak, M. (1975). A comprehensive introduction to differential geometry, vol. 1--5. Publish or Perish.