Evolving neural network ensembles for control problems

ABSTRACT

In neuroevolution, a genetic algorithm is used to evolve a neural network to perform a particular task. The standard approach is to evolve a population over a number of generations, and then select the final generation's champion as the end result. However, it is possible that there is valuable information present in the population that is not captured by the champion. The standard approach ignores all such information. One possible solution to this problem is to combine multiple individuals from the final population into an ensemble. This approach has been successful in supervised classification tasks, and in this paper, it is extended to evolutionary reinforcement learning in control problems. The method is evaluated on a challenging extension of the classic pole balancing task, demonstrating that an ensemble can achieve significantly better performance than the champion alone.

REFERENCES

• 1: Andrew G. Barto , Sridhar Mahadevan, Recent Advances in Hierarchical Reinforcement Learning, Discrete Event Dynamic Systems, v.13 n.4, p.341-379, October 2003 [doi>10.1023/A:1025696116075]
• 2: Leo Breiman, Bagging predictors, Machine Learning, v.24 n.2, p.123-140, Aug. 1996 [doi>10.1023/A:1018054314350]
• 3: J. Bruce and R. Miikkulainen. Evolving populations of expert neural networks. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), pages 251--257, San Francisco, CA, 2001.
• 4: Y. Freund and R. E. Schapire. Experiments with a new boosting algorithm. In International Conference on Machine Learning, pages 148--156, 1996.
• 5: F. Gomez and R. Miikkulainen. 2-d pole balancing with recurrent evolutionary networks. In Proceedings of the International Conference on Artificial Neural Networks (ICANN-98, Skovde, Sweden), pages 425--430, 1998.
• 6: R. A. Jacobs, M. I. Jordan, S. Nowlan, and G. E. Hinton. Adaptive mixtures of local experts. Neural Computation, 3:79 -- 87, 1991.
• 7: A. Krogh and J. Vedelsby. Neural network ensembles, cross validation, and active learning. In Advances in Neural Information Processing Systems, volume 7, pages 231--238. The MIT Press, 1995.
• 8: Yong Liu , Xin Yao, Towards Designing Neural Network Ensembles by Evolution, Proceedings of the 5th International Conference on Parallel Problem Solving from Nature, p.623-632, September 27-30, 1998
• 9: D. O. N. García, C. Hervás. Cooperative coevolution of artificial neural network ensembles for pattern classification. IEEE Transactions on Evolutionary Computation, 2005. In press.
• 10: D. W. Opitz and J. W. Shavlik. Generating accurate and diverse members of a neural-network ensemble. In Advances in Neural Information Processing Systems, volume 8, pages 535--541. The MIT Press, 1996.
• 11: J. D. Schaffer, D. Whitley, and L. J. Eshelman. Combinations of genetic algorithms and neural networks: A survey of the state of the art. In Proceedings of the International Workshop on Combinations of Genetic Algorithms and Neural Networks (COGANN-92), pages 1--37, Los Alamitos, CA, 1992. IEEE Press, Piscataway, New Jersey.
• 12: Kenneth O. Stanley , Risto Miikkulainen, Evolving neural networks through augmenting topologies, Evolutionary Computation, v.10 n.2, p.99-127, Summer 2002 [doi>10.1162/106365602320169811]
• 13: X. Yao. Evolving artificial neural networks. Proceedings of the IEEE, 87(9):1423--1447, 1999.
• 14: X. Yao and Y. Liu. Making use of population information in evolutionary artificial neural networks. IEEE Transactions on Systems, Man, and Cybernetics, 28B(2), April 1998.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning

General Terms:
Algorithms, Experimentation

Keywords:
ensembles, genetic algorithms, neural networks, reinforcement learning