Modeling UCS as a mixture of experts

ABSTRACT

We present a probabilistic formulation of UCS (a sUpervised Classifier System). UCS is shown to be a special case of mixture of experts where the experts are learned independently and later combined during prediction. In this work, we develop the links between the constituent components of UCS and a mixture of experts, thus lending UCS a strong analytical background. We find during our analysis that mixture of experts is a more generic formulation of UCS and possesses more generalization capability and flexibility than UCS, which is also verified using empirical evaluations. This is the first time that a simple probabilistic model has been proposed for UCS and we believe that this work will form a useful tool to analyse Learning Classifier Systems and gain useful insights into their working.

REFERENCES

• 1: Ester Bernadó-Mansilla , Josep M. Garrell-Guiu, Accuracy-based learning classifier systems: models, analysis and applications to classification tasks, Evolutionary Computation, v.11 n.3, p.209-238, Fall 2003 [doi>10.1162/106365603322365289]
• 2: Gavin Brown , Jeremy L. Wyatt , Peter Tiňo, Managing Diversity in Regression Ensembles, The Journal of Machine Learning Research, 6, p.1621-1650, 12/1/2005
• 3: Arthur Dempster, Nan Laird, and Donald Rubin. Likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39(1):1--38, 1977.
• 4: Jan Drugowitsch, Design and Analysis of Learning Classifier Systems: A Probabilistic Approach (Studies in Computational Intelligence), Springer Publishing Company, Incorporated, 2008
• 5: Narayanan U. Edakunni and Tim Kovacs. Probabilistic modeling of UCS : a theoretical study. Technical report, University of Bristol, 2009.
• 6: Robert A. Jacobs , Michael I. Jordan , Steven J. Nowlan , Geoffrey E. Hinton, Adaptive mixtures of local experts, Neural Computation, v.3 n.1, p.79-87, Spring 1991 [doi>10.1162/neco.1991.3.1.79]
• 7: Tim Kovacs. Genetics-based machine learning. In Grzegorz Rozenberg, Thomas Back, and Joost Kok, editors, Handbook of Natural Computing: Theory, Experiments, and Applications. Springer Verlag, 2009.
• 8: James A. R. Marshall , Gavin Brown , Tim Kovacs, Bayesian estimation of rule accuracy in UCS, Proceedings of the 2007 GECCO conference companion on Genetic and evolutionary computation, July 07-11, 2007, London, United Kingdom [doi>10.1145/1274000.1274066]

INDEX TERMS

Primary Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS

General Terms:
Algorithms

Keywords:
UCS, learning classifier system, mixture of experts, probabilistic modeling