MILCS: a mutual information learning classifier system

ABSTRACT

This paper introduces a new variety of learning classifier system (LCS), called MILCS, which utilizes mutual information as fitness feedback. Unlike most LCSs, MILCS is specifically designed for supervised learning. MILCS's design draws on an analogy to the structural learning approach of cascade correlation networks. We present preliminary results, and contrast them to results from XCS. We discuss the explanatory power of the resulting rule sets, and introduce a new technique for visualizing explanatory power. Final comments include future directions for this research, including investigations in neural networks and other systems.

REFERENCES

• 1: Bacardit, J. Pittsburgh Genetics-Based Machine Learning in the Data Mining era: representations, generalization, and run-time (2004). PhD thesis, Ramon Llull University, Barcelona, Catalonia, Spain.
• 2: Ester Bernadó-Mansilla , Josep M. Garrell-Guiu, Accuracy-based learning classifier systems: models, analysis and applications to classification tasks, Evolutionary Computation, v.11 n.3, p.209-238, Fall 2003 [doi>10.1162/106365603322365289]
• 3: Butz, M. V.(203) Documentation of XCS+TS C-Code 1.2. IlliGAL report 2003023, University of Illinois at Urbana-Champaign. (Source code: ftp://gal2.ge.uiuc.edu/pub/src/XCS/XCS1.2.tar.Z).
• 4: (2003) ftp://gal2.ge.uiuc.edu/pub/src/XCS/XCS1.2.tar.ZXCS (+ tournament selection) classifier system implementation in C, version 1.2 (ftp://gal2.ge.uiuc.edu/pub/src/XCS/XCS1.2.tar.Z) (for IlliGAL Report 2003023, University of Illinois Genetic Algorithms Laboratory).
• 5: Scott E. Fahlman , Christian Lebiere, The cascade-correlation learning architecture, Advances in neural information processing systems 2, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1990
• 6: David E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1989
• 7: Heckerman, H., (1996). A Tutorial on Learning with Bayesian Networks, Technical Report, MSR-TR-95-06.
• 8: John H. Holland, Adaptation in natural and artificial systems, MIT Press, Cambridge, MA, 1992
• 9: John H. Holland , Keith J. Holyoak , Richard E. Nisbett , Paul R. Thagard, Induction: processes of inference, learning, and discovery, MIT Press, Cambridge, MA, 1986
• 10: Lanzi, P. L. xcslib: source code available at http://xcslib.sourceforge.net/.
• 11: Shannon, C.E. (1948). A Mathematical Theory of Communication, Bell System Technical Journal, 27, pp. 379--423 & 623--656, July & October, 1948.
• 12: Shannon, C.E. (1949). Communication in the presence of noise, Proc. Institute of Radio Engineers, vol. 37, no.1, pp. 10--21, Jan. 1949.
• 13: Smith, R. E. and Cribbs, H. B. (1994). Is a classifier system a type of neural network? Evolutionary Computation, 2(1), 19--36.
• 14: Richard S. Sutton , Andrew G. Barto, Introduction to Reinforcement Learning, MIT Press, Cambridge, MA, 1998
• 15: Stout, M., Bacardit, J., Hirst, J., Krasogor, N. and Blazewicz, J. (2006). From HP lattice models to real proteins: coordination number prediction using learning classifier systems. In 4th European Workshop on Evolutionary Computation and Machine Learning in Bioinformatics 2006.
• 16: Wilson, S. W. (1994). Classifier Fitness based on Accuracy, Evolutionary Computation 3(2). pp 149--175.
• 17: Wilson, S. W. (1994). ZCS: A Zeroth-Level Classifier System, Evolutionary Computation 2(1). pp 1--18.
• 18: Wilson, S. W. (1998) Generalization in the XCS classifier system. In Koza, J. R., Banzhaf, W., Chellapilla, K., Deb, K., Dorigo, M., Fogel, D. B., Garzon, M. H., Goldberg, D. E., Iba, H., and Riolo, R. (eds.), Genetic Programming 1998: Proceedings of the Third Annual Conference, pages 665--674. Morgan Kaufmann.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
          Subjects: Induction

General Terms:
Algorithms

Keywords:
cascade correlation, evolutionary computation, explanatory power, information theory, learning classifier systems, mutual information, rule learning, structural learning, supervised learning, visualization