Evolving competitive car controllers for racing games with neuroevolution

ABSTRACT

Modern computer games are at the same time an attractive application domain and an interesting testbed for the evolutionary computation techniques. In this paper we apply NeuroEvolution of Augmenting Topologies (NEAT), a well known neuroevolution approach, to evolve competitive non-player characters for a racing game. In particular, we focused on The Open Car Racing Simulator (TORCS), an open source car racing simulator, already used as a platform for several scientific competitions dedicated to games. We suggest that a competitive controller should have two basic skills: it should be able to drive fast and reliably on a wide range of tracks and it should be able to effectively overtake the opponents avoiding the collisions. In this paper we apply NEAT to evolve separately these skills and then we combined them together in a single controller. Our results show that the resulting controller outperforms the best available controllers on a challenging racing task. In addition, the experimental analysis also confirms that both the skills are necessary to develop a competitive controller.

REFERENCES

• 1: The open racing car simulator website. Online. http://torcs.sourceforge.net/.
• 2: Benoit Chaperot and Colin Fyfe. Improving artificial intelligence in a motocross game. In IEEE Symposium on Computational Intelligence and Games, 2006.
• 3: S. A. Glantz and B. K. Slinker. Primer of Applied Regression&Analysis of Variance. McGraw Hill, 2001. second edition.
• 4: David E. Goldberg , Jon Richardson, Genetic algorithms with sharing for multimodal function optimization, Proceedings of the Second International Conference on Genetic Algorithms on Genetic algorithms and their application, p.41-49, October 1987, Cambridge, Massachusetts, United States
• 5: Jeff Hannan. Interview to jeff hannan, 2001. http://www.generation5.org/content/2001/hannan.asp.
• 6: Daniele Loiacono, Julian Togelius, and Pier Luca Lanzi. The car racing competition homepage. Online. http://cig.dei.polimi.it/.
• 7: Daniele Loiacono, Julian Togelius, and Pier Luca Lanzi. Software manual of the car racing competition. Online. http://mesh.dl.sourceforge.net/sourceforge/cig/CIG2008-Manual-V1.pdf.
• 8: Daniele Loiacono, Julian Togelius, Pier Luca Lanzi, Leonard Kinnaird-Heether, Simon M. Lucas, Matt Simmerson, Diego Perez, Robert G. Reynolds, and Yago Saez. The wcci 2008 simulated car racing competition. In Proceedings of the IEEE Symposium on Computational Intelligence and Games, 2008.
• 9: Simon M. Lucas, Ms Pac-Man competition, ACM SIGEVOlution, v.2 n.4, p.37-38, Winter 2007 [doi>10.1145/1399962.1399969]
• 10: Simon M. Lucas and Thomas P. Runarsson. Temporal difference learning versus co-evolution for acquiring othello position evaluation. In IEEE Symposium on Computational Intelligence and Games (CIG), 2006.
• 11: Steffen Priesterjahn, Kramer, Alexander Weimer, and Andreas Goebels. Evolution of human-competitive agents in modern computer games. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC), 2007.
• 12: Larry D. Pyeatt , Adele E. Howe, Learning to Race: Experiments with a Simulated Race Car, Proceedings of the Eleventh International Florida Artificial Intelligence Research Society Conference, p.357-361, May 18-20, 1998
• 13: R.G. Reynolds and M. Ali. Computing with the social fabric: The evolution of social intelligence within a cultural framework. Computational Intelligence Magazine, IEEE, 3(1):18--30, February 2008.
• 14: Matt Simmerson. Neat4j homepage. Online, 2006. 2006. {Online}. Available: http://neat4j.sourceforge.net.
• 15: Kenneth O. Stanley, Bobby D. Bryant, and Risto Miikkulainen. Real-time neuroevolution in the nero video game. IEEE Transactions on Evolutionary Computation, 9(6):653--668, 2005.
• 16: Kenneth Stanley , Nate Kohl , Rini Sherony , Risto Miikkulainen, Neuroevolution of an automobile crash warning system, Proceedings of the 2005 conference on Genetic and evolutionary computation, June 25-29, 2005, Washington DC, USA [doi>10.1145/1068009.1068340]
• 17: Kenneth O. Stanley , Risto Miikkulainen, Evolving neural networks through augmenting topologies, Evolutionary Computation, v.10 n.2, p.99-127, Summer 2002 [doi>10.1162/106365602320169811]
• 18: Julian Togelius. Optimization, Imitation and Innovation: Computational Intelligence and Games. PhD thesis, Department of Computing and Electronic Systems, University of Essex, Colchester, UK, 2007.
• 19: Julian Togelius and Simon M. Lucas. Evolving controllers for simulated car racing. In Proceedings of the Congress on Evolutionary Computation, 2005.
• 20: Julian Togelius and Simon M. Lucas. Arms races and car races. In Proceedings of Parallel Problem Solving from Nature. Springer, 2006.
• 21: Julian Togelius and Simon M. Lucas. Evolving robust and specialized car racing skills. In Proceedings of the IEEE Congress on Evolutionary Computation, 2006.
• 22: Julian Togelius , Simon Lucas , Ho Duc Thang , Jonathan M. Garibaldi , Tomoharu Nakashima , Chin Hiong Tan , Itamar Elhanany , Shay Berant , Philip Hingston , Robert M. Maccallum , Thomas Haferlach , Aravind Gowrisankar , Pete Burrow, The 2007 IEEE CEC simulated car racing competition, Genetic Programming and Evolvable Machines, v.9 n.4, p.295-329, December 2008 [doi>10.1007/s10710-008-9063-0]
• 23: Krzysztof Wloch and Peter J. Bentley. Optimising the performance of a formula one car using a genetic algorithm. In Proceedings of Eighth International Conference on Parallel Problem Solving From Nature, pages 702--711, 2004.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.1 Applications and Expert Systems
          Subjects: Games

General Terms:
Algorithms, Experimentation, Performance

Keywords:
games, neat, simulated car racing, torcs