Automatic feature selection in neuroevolution

ABSTRACT

Feature selection is the process of finding the set of inputs to a machine learning algorithm that will yield the best performance. Developing a way to solve this problem automatically would make current machine learning methods much more useful. Previous efforts to automate feature selection rely on expensive meta-learning or are applicable only when labeled training data is available. This paper presents a novel method called FS-NEAT which extends the NEAT neuroevolution method to automatically determine an appropriate set of inputs for the networks it evolves. By learning the network's inputs, topology, and weights simultaneously, FS-NEAT addresses the feature selection problem without relying on meta-learning or labeled data. Initial experiments in an autonomous car racing simulation demonstrate that FS-NEAT can learn better and faster than regular NEAT. In addition, the networks it evolves are smaller and require fewer inputs. Furthermore, FS-NEAT's performance remains robust even as the feature selection task it faces is made increasingly difficult.

REFERENCES

• 1: B. V. Bonnlander and A. S. Weigend. Selecting input variables using mutual information and nonparametric density estimation. In Proceedings of the 1994 International Symposium on Artificial Neural Networks (ISANN'94), pages 42--50, Tainan, Taiwan, 1994.
• 2: David E. Goldberg , Jon Richardson, Genetic algorithms with sharing for multimodal function optimization, Proceedings of the Second International Conference on Genetic Algorithms on Genetic algorithms and their application, p.41-49, October 1987, Cambridge, Massachusetts, United States
• 3: F. Gruau, D. Whitley, and L. Pyeatt. A comparison between cellular encoding and direct encoding for genetic neural networks. In J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo, editors, Genetic Programming 1996: Proceedings of the First Annual Conference, pages 81--89. MIT Press, 1996.
• 4: P. R. Harvey, D. M. Booth, and J. F. Boyce. Evolving the mapping between input neurons and multi-source imagery. In Proceedings of the 2002 Congress on Evolutionary Computation, pages 1878--1883, 2002.
• 5: Kenji Kira , Larry A. Rendell, A Practical Approach to Feature Selection, Proceedings of the Ninth International Workshop on Machine Learning, p.249-256, July 01-03, 1992
• 6: P. Langley. Selection of relevant features in machine learning. In Proceedings of AAAI Fall Symposium on Relevance, 1994.
• 7: P. M. Narendra and K. Fukunaga. A branch and bound algorithm for feature subset selection. IEEE Transactions on Computers, 26:917--922, 1977.
• 8: P. Pudil , J. Novovičová , J. Kittler, Floating search methods in feature selection, Pattern Recognition Letters, v.15 n.11, p.1119-1125, Nov. 1994 [doi>10.1016/0167-8655(94)90127-9]
• 9: N. J. Radcliffe. Genetic set recombination and its application to neural network topology optimization. Neural computing and applications, 1(1):67--90, 1993.
• 10: Kenneth O. Stanley , Risto Miikkulainen, Evolving neural networks through augmenting topologies, Evolutionary Computation, v.10 n.2, p.99-127, Summer 2002 [doi>10.1162/106365602320169811]
• 11: K. O. Stanley and R. Miikkulainen. Competitive coevolution through evolutionary complexification. Journal of Artificial Intelligence Research, 21, 2004. In press.
• 12: M. E. Timin. The robot auto racing simulator, 1995. http://rars.sourceforge.net.
• 13: S. Whiteson, K. O. Stanley, and R. Miikkulainen. Automatic feature selection in neuroevolution. In GECCO 2004: Proceedings of the Genetic and Evolutionary Computation Conference Workshop on Self-Organization, July 2004.
• 14: X. Yao. Evolving artificial neural networks. Proceedings of the IEEE, 87(9):1423--1447, 1999.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
          Subjects: Connectionism and neural nets

General Terms:
Experimentation

Keywords:
feature selection, genetic algorithms, neural networks