Towards efficient online reinforcement learning using neuroevolution

ABSTRACT

For many complex Reinforcement Learning (RL) problems with large and continuous state spaces, neuroevolution has achieved promising results. This is especially true when there is noise in sensor and/or actuator signals. These results have mainly been obtained in offline learning settings, where the training and the evaluation phases of the systems are separated. In contrast, for online RL tasks, the actual performance of a system matters during its learning phase. In these tasks, neuroevolutionary systems are often impaired by their purely exploratory nature, meaning that they usually do not use (i.e. exploit) their knowledge of a single individual's performance to improve performance during learning. In this paper we describe modifications that significantly improve the online performance of the neuroevolutionary method Evolutionary Acquisition of Neural Topologies and discuss the results obtained in the Mountain Car benchmark.

REFERENCES

• 1: Richard S. Sutton , Andrew G. Barto, Introduction to Reinforcement Learning, MIT Press, Cambridge, MA, 1998
• 2: Shimon Whiteson , Matthew E. Taylor , Peter Stone, Empirical Studies in Action Selection with Reinforcement Learning, Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems, v.15 n.1, p.33-50, March 2007 [doi>10.1177/1059712306076253]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
          Subjects: Connectionism and neural nets

General Terms:
Algorithms

Keywords:
neuroevolution, online-learning, reinforcement learning