Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

In recent years, neuroevolutionary methods have shown great promise in solving learning tasks, especially in domains that are stochastic, partially observable, and noisy. In this paper, we show how the Kalman filter can be exploited (1) to efficiently find an optimal solution (i. e. reducing the number of evaluations needed to find the solution), (2) to find solutions that are robust against noise, and (3) to recover or reconstruct missing state variables, traditionally known as state estimation in control engineering community. Our algorithm has been tested on the double pole balancing without velocities benchmark, and has achieved significantly better results on this benchmark than the published results of other algorithms to date.

REFERENCES

• 1: Yaakov Bar-Shalom , Xiao-Rong Li, Estimation with Applications to Tracking and Navigation, John Wiley & Sons, Inc., New York, NY, 2001
• 2: P. Dürr, C. Mattiussi, and D. Floreano. Neuroevolution with analog genetic encoding. In Proceedings of the 9th Conference on Parallel Problem Solving from Nature (PPSN IX), pages 671--680, 2006.
• 3: Faustino Gomez , Risto Mikkulainen, Incremental evolution of complex general behavior, Adaptive Behavior, v.5 n.3-4, p.317-342, Winter/Spring 1997 [doi>10.1177/105971239700500305]
• 4: F. J. Gomez, J. Schmidhuber, and R. Miikkulainen. Efficient non-linear control through neuroevolution. In Proceedings of the European Conference on Machine Learning (ECML 2006), pages 654--662, 2006.
• 5: F. Gruau. Neural Network Synthesis Using Cellular Encoding and the Genetic Algorithm. PhD thesis, Ecole Normale Superieure de Lyon, Laboratoire de l'Informatique du Parallelisme, France, January 1994.
• 6: F. Gruau, D. Whitley, and L. Pyeatt. A comparison between cellular encoding and direct encoding for genetic neural networks. In J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo, editors, Genetic Programming: Proceedings of the First Annual Conference, pages 81--89, Standford University, CA, USA, 1996. MIT Press.
• 7: Nikolaus Hansen , Andreas Ostermeier, Completely Derandomized Self-Adaptation in Evolution Strategies, Evolutionary Computation, v.9 n.2, p.159-195, June 2001 [doi>10.1162/106365601750190398]
• 8: C. Igel. Neuroevolution for reinforcement learning using evolution strategies. In R. Sarker, R. Reynolds, H. Abbass, K. C. Tan, B. McKay, D. Essam, and T. Gedeon, editors, Congress on Evolutionary Computation (CEC2003), volume 4, pages 2588--2595. IEEE Press, 2003.
• 9: L. P. Kaelbling, M. L. Littman, and A. P. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237--285, 1996.
• 10: P. R. Kalata. Alpha-beta target tracking systems: A survey. In American Control Conference, pages 832--836, 1992.
• 11: R. E. Kalman. A new approach to linear filtering and prediction problems. Transactions of the ASME-Journal of Basic Engineering, Series D:35--45, 1960.
• 12: Yohannes Kassahun , Mark Edgington , Jan Hendrik Metzen , Gerald Sommer , Frank Kirchner, A common genetic encoding for both direct and indirect encodings of networks, Proceedings of the 9th annual conference on Genetic and evolutionary computation, July 07-11, 2007, London, England [doi>10.1145/1276958.1277162]
• 13: Y. Kassahun and G. Sommer. Efficient reinforcement learning through evolutionary acquisition of neural topologies. In Proceedings of the 13th European Symposium on Artificial Neural Networks (ESANN 2005), pages 259--266, Bruges, Belgium, April 2005.
• 14: Long Lin , Tom Mitchell, Memory Approaches to Reinforcement Learning in Non-Markovian Domains, Carnegie Mellon University, Pittsburgh, PA, 1992
• 15: David E. Moriarty , Risto Mikkulainen, Efficient reinforcement learning through symbiotic evolution, Machine Learning, v.22 n.1-3, p.11-32, Jan./Feb./March 1996 [doi>10.1007/BF00114722]
• 16: N. Saravanan , David B. Fogel, Evolving Neural Control Systems, IEEE Expert: Intelligent Systems and Their Applications, v.10 n.3, p.23-27, June 1995 [doi>10.1109/64.393139]
• 17: Hans-Paul Paul Schwefel, Evolution and Optimum Seeking: The Sixth Generation, John Wiley & Sons, Inc., New York, NY, 1993
• 18: Nils T. Siebel , Gerald Sommer, Evolutionary reinforcement learning of artificial neural networks, International Journal of Hybrid Intelligent Systems, v.4 n.3, p.171-183, August 2007
• 19: Kenneth Owen Stanley , Risto P. Miikkulainen, Efficient evolution of neural networks through complexification, The University of Texas at Austin, 2004
• 20: Greg Welch , Gary Bishop, An Introduction to the Kalman Filter, University of North Carolina at Chapel Hill, Chapel Hill, NC, 1995
• 21: A. Wieland. Evolving controls for unstable systems. In Proceedings of the International Joint Conference on Neural Networks, pages 667--673, 1991.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
          Subjects: Connectionism and neural nets

General Terms:
Algorithms

Keywords:
Kalman filter, neuroevolution