Efficient natural evolution strategies

ABSTRACT

Efficient Natural Evolution Strategies (eNES) is a novel alternative to conventional evolutionary algorithms, using the natural gradient to adapt the mutation distribution. Unlike previous methods based on natural gradients, eNES uses a fast algorithm to calculate the inverse of the exact Fisher information matrix, thus increasing both robustness and performance of its evolution gradient estimation, even in higher dimensions. Additional novel aspects of eNES include optimal fitness baselines and importance mixing (a procedure for updating the population with very few fitness evaluations). The algorithm yields competitive results on both unimodal and multimodal benchmarks.

REFERENCES

• 1: Shun-Ichi Amari, Natural gradient works efficiently in learning, Neural Computation, v.10 n.2, p.251-276, Feb. 15, 1998 [doi>10.1162/089976698300017746]
• 2: S. Amari and S. C. Douglas. Why natural gradient? volume 2, pages 1213--1216 vol.2, 1998.
• 3: Nikolaus Hansen , Andreas Ostermeier, Completely Derandomized Self-Adaptation in Evolution Strategies, Evolutionary Computation, v.9 n.2, p.159-195, June 2001 [doi>10.1162/106365601750190398]
• 4: Jan Peters , Stefan Schaal, Natural Actor-Critic, Neurocomputing, v.71 n.7-9, p.1180-1190, March, 2008 [doi>10.1016/j.neucom.2007.11.026]
• 5: Jan Reinhard Peters , Stefan Schaal, Machine learning of motor skills for robotics, University of Southern California, Los Angeles, CA, 2007
• 6: P. N. Suganthan, N. Hansen, J. J. Liang, K. Deb, Y. P. Chen, A. Auger, and S. Tiwari. Problem definitions and evaluation criteria for the cec 2005 special session on real-parameter optimization. Technical report, Nanyang Technological University, Singapore, 2005.
• 7: Sun Yi , Daan Wierstra , Tom Schaul , Jürgen Schmidhuber, Stochastic search using the natural gradient, Proceedings of the 26th Annual International Conference on Machine Learning, p.1161-1168, June 14-18, 2009, Montreal, Quebec, Canada [doi>10.1145/1553374.1553522]
• 8: D. Wierstra, T. Schaul, J. Peters, and J. Schmidhuber. Natural evolution strategies. In Proceedings of the Congress on Evolutionary Computation (CEC08), Hongkong. IEEE Press, 2008.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.8 Problem Solving, Control Methods, and Search
          Subjects: Heuristic methods

Additional Classification:
  G. Mathematics of Computing
  G.1 NUMERICAL ANALYSIS
      G.1.6 Optimization
          Subjects: Global optimization

General Terms:
Algorithms

Keywords:
evolution strategies, natural gradient, optimization