XCSF with computed continuous action

ABSTRACT

Wilson introduced XCSF as a successor to XCS. The major development of XCSF is the concept of a computed prediction. The efficiency of XCSF in dealing with numerical input and continuous payoff has been demonstrated. However, the possible actions must always be determined in advance. Yet domains such as robot control require numerical actions, so that neither XCS nor XCSF with their discrete actions can yield high performance. This paper studies computed action in XCSF, where the action is continuous with respect to the input state. In comparison with Wilson's architecture for continuous action, our XCSF version, called XCSFCA, proves to be more efficient.

REFERENCES

• 1: Ronald C. Arkin, Behavior-based robot navigation for extended domains, Adaptive Behavior, v.1 n.2, p.201-225, Fall 1992 [doi>10.1177/105971239200100204]
• 2: Thomas Bäck, Evolutionary algorithms in theory and practice: evolution strategies, evolutionary programming, genetic algorithms, Oxford University Press, Oxford, 1996
• 3: Butz, M. V., and Wilson, S. W., An Algorithmic Description of XCS, Soft Computing, 6(3-4), pp. 144--153, 2002.
• 4: Lanzi, P. L., Loiacono, D., Wilson, S. W., and Goldberg, D. E., XCS with Computed Prediction for the Learning of Boolean Functions, Proceedings of the IEEE Congress on Evolutionary Computation Conference (CEC2005), 2005.
• 5: Sanchez, S., Mécanismes Evolutionnistes pour la Simulation Comportementale d'Acteurs Virtuels, PhD thesis, pp. 93--94, University Toulouse I, IRIT (Institut de Recherche en Informatique de Toulouse), France, 2004.
• 6: Hans-Paul Schwefel, Numerical Optimization of Computer Models, John Wiley & Sons, Inc., New York, NY, 1981
• 7: Richard S. Sutton , Andrew G. Barto, Introduction to Reinforcement Learning, MIT Press, Cambridge, MA, 1998
• 8: Tran, T. H., Sanza, C., and Duthen, Y., Evolving the Motor Schema Approach for Learning Cooperation, CASA'2006 The nineteenth Conference on Computer Animation and Social Agents, pp. 219--228, Geneve, Suisse, 2006.
• 9: Bernard Widrow , Marcian E. Hoff, Adaptive switching circuits, Neurocomputing: foundations of research, MIT Press, Cambridge, MA, 1988
• 10: Wilson, S. W., ZCS: A Zeroth Level Classifier System, Evolutionary Computation, 2(1), pp. 1--18, 1994.
• 11: Wilson, S. W., Classifier Fitness Based on Accuracy, Evolutionary Computation, 3(2), pp. 149--175, 1995.
• 12: Wilson, S. W., Get Real! XCS with Continuous-Valued Inputs, From Festschrift in Honor of John H. Holland, L. Booker, S. Forrest, M. Mitchell, and R. Riolo (eds.), pp. 111--121, 1999.
• 13: Wilson, S. W., Function Approximation with a Classifier System, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), L. Spector et al, eds. Morgan Kaufmann, San Francisco, CA, pp. 974--981, 2001.
• 14: Stewart W. Wilson, Classifiers that approximate functions, Natural Computing: an international journal, v.1 n.2-3, p.211-234, June 2002 [doi>10.1023/A:1016535925043]
• 15: Wilson, S. W., Classifier Systems for Continuous Payoff Environments, Proceedings of the Genetic and Evolutionary Computation Conference (GECCO--2004), K. Deb et al, eds. Springer--Verlag, Berlin, pp. 824--835 in Part II, 2004.
• 16: Wilson, S. W., Three Architectures for Continuous Action, Technical Report No. 2006019, Illinois Genetic Algorithms Laboratory, University of Illinois at Urbana--Champaign, 2006.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
          Subjects: Concept learning

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.8 Problem Solving, Control Methods, and Search
          Subjects: Heuristic methods

General Terms:
Algorithms

Keywords:
XCSF, continuous actions, evolution strategy