Unsupervised learning of echo state networks: balancing the double pole

ABSTRACT

A possible alternative to fine topology tuning for Neural Network (NN) optimization is to use Echo State Networks (ESNs), recurrent NNs built upon a large reservoir of sparsely randomly connected neurons. The promises of ESNs have been fulfilled for supervised learning tasks, but unsupervised learning tasks, such as control problems, require more flexible optimization methods. We propose here to apply state-of-the-art methods in evolutionary continuous parameter optimization, to the evolutionary learning of ESN. First, a standard supervised learning problem is used to validate our approach and compare it to the standard quadratic one. The classical double pole balancing control problem is then used to demonstrate that unsupervised evolutionary learning of ESNs yields results that compete with the best topology-learning methods.

REFERENCES

• 1: P. Dürr, C. Mattiussi, and D. Floreano. Neuroevolution with Analog Genetic Encoding. In Th. Runarsson et al., editor, PPSN IX, pages 671--680, 2006.
• 2: N. Hansen and S. Kern. Evaluating the CMA evolution strategy on multimodal test functions. In X. Yao et al., editors, PPSN VIII, pages 282--291, 2004.
• 3: C. Igel. Neuroevolution for reinforcement learning using evolution strategies. In Proc. CEC'03, pages 2588--2595. IEEE Press, 2003.
• 4: H. Jaeger. The Echo State Approach to Analysing and Training Recurrent Neural Networks. Technical Report GMD 148, German National Research Center for Information Technology, 2001.
• 5: Kenneth O. Stanley , Risto Miikkulainen, Efficient Reinforcement Learning Through Evolving Neural Network Topologies, Proceedings of the Genetic and Evolutionary Computation Conference, p.569-577, July 09-13, 2002

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
          Subjects: Connectionism and neural nets

General Terms:
Algorithms, Design

Keywords:
CMA-ES, echo state networks, evolution strategies