Modular neuroevolution for multilegged locomotion

ABSTRACT

Legged robots are useful in tasks such as search and rescue because they can effectively navigate on rugged terrain. However, it is difficult to design controllers for them that would be stable and robust. Learning the control behavior is difficult because optimal behavior is not known, and the search space is too large for reinforcement learning and for straightforward evolution. As a solution, this paper proposes a modular approach for evolving neural network controllers for such robots. The search space is effectively reduced by exploiting symmetry in the robot morphology, and encoding it into network modules. Experiments involving physically realistic simulations of a quadruped robot produce the same symmetric gaits, such as pronk, pace, bound and trot, that are seen in quadruped animals. Moreover, the robot can transition dynamically to more effective gaits when faced with obstacles. The modular approach also scales well when the number of legs or their degrees of freedom are increased. Evolved non-modular controllers, in contrast, produce gaits resembling crippled animals that are much less effective and do not scale up as a result. Hand-designed controllers are also less effective, especially on an obstacle terrain. These results suggest that the modular approach is effective for designing robust locomotion controllers for multilegged robots.

REFERENCES

• 1: Randall D. Beer , Hillel J. Chiel , Leon S. Sterling, Heterogeneous neural networks for adaptive behavior in dynamic environments, Advances in neural information processing systems 1, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1989
• 2: Randall D. Beer , John C. Gallagher, Evolving dynamical neural networks for adaptive behavior, Adaptive Behavior, v.1 n.1, p.91-122, Summer 1992 [doi>10.1177/105971239200100105]
• 3: A. Billard and A. J. Ijspeert. Biologically inspired neural controllers for motor control in a quadruped robot. In Proceedings of IJCNN-2000, pages 637--641, 2000.
• 4: Rodney A. Brooks, A Robot That Walks; Emergent Behaviors from a Carefully Evolved Network, Massachusetts Institute of Technology, Cambridge, MA, 1989
• 5: Jonathan E. Clark , Mark R. Cutkosky, Design, simulation, and stability of a hexapedal running robot, Stanford University, Stanford, CA, 2004
• 6: J. J. Collins and I. N. Stewart. Coupled nonlinear oscillators and the symmetries of animal gaits. Journal of Nonlinear Science, 3(1):349--392, 1993.
• 7: D. Filliat, J. Kodjabachian, and J.-A. Meyer. Evolution of neural controllers for locomotion and obstacle avoidance in a six-legged robot. Connection Science, 11(3/4):225--242, 1999.
• 8: D. Floreano , F. Mondada, Evolutionary neurocontrollers for autonomous mobile robots, Neural Networks, v.11 n.7-8, p.1461-1478, Oct./Nov. 1998 [doi>10.1016/S0893-6080(98)00082-3]
• 9: M. Golubitsky and I. Stewart. Patterns of oscillation in coupled cell systems. In Geometry, Mechanics, and Dynamics: Volume in Honor of the 60th Birthday of J. E. Marsden, chapter 8, pages 243--286, 2002.
• 10: F. Gomez and R. Miikkulainen. Active guidance for a finless rocket using neuroevolution. In Proceedings of GECCO-2003, pages 2084--2095, 2003.
• 11: F. J. Gomez, J. Schmidhuber, and R. Miikkulainen. Efficient non-linear control through neuroevolution. In Proceedings of ECML-2006, pages 654--662, 2006.
• 12: Frédéric Gruau, Automatic definition of modular neural networks, Adaptive Behavior, v.3 n.2, p.151-183, Fall 1994 [doi>10.1177/105971239400300202]
• 13: F. Gruau. Neural Network Synthesis Using Cellular Encoding and the Genetic Algorithm. PhD thesis, Ecole Normale Superieure de Lyon, France, 1994.
• 14: Philip Holmes , Robert J. Full , Dan Koditschek , John Guckenheimer, The Dynamics of Legged Locomotion: Models, Analyses, and Challenges, SIAM Review, v.48 n.2, p.207-304, 2006 [doi>10.1137/S0036144504445133]
• 15: Gregory S. Hornby , Jordan B. Pollack, Creating high-level components with a generative representation for body-brain evolution, Artificial Life, v.8 n.3, p.223-246, 2002 [doi>10.1162/106454602320991837]
• 16: D. E. Koditschek, R. J. Full, and M. Buehler. Mechanical aspects of legged locomotion control. Arthropod Structure and Development, 33(3):251--272, July 2004.
• 17: D. E. Moriarty and R. Miikkulainen. Hierarchical evolution of neural networks. In Proceedings of the 1998 IEEE Conference on Evolutionary Computation, pages 428--433, 1998.
• 18: Fionn Murtagh, Clustering in massive data sets, Handbook of massive data sets, Kluwer Academic Publishers, Norwell, MA, 2002
• 19: ODE, 2007. \beginsmallhttp://www.ode.org\endsmall.
• 20: OGRE, 2007. \beginsmallhttp://www.ogre3d.org\endsmall.
• 21: OPAL, 2007. \beginsmallhttp://opal.sourceforge.net\endsmall.
• 22: Open BEAGLE, 2007. \beginsmallhttp://beagle.gel.ulaval.ca\endsmall.
• 23: C. M. A. Pinto and M. Golubitsky. Central pattern generators for bipedal locomotion. Journal of Mathematical Biology, 53(3):474--489, 2006.
• 24: Marc H. Raibert, Legged robots, Communications of the ACM, v.29 n.6, p.499-514, June 1986 [doi>10.1145/5948.5950]
• 25: M. H. Raibert, M. Chepponis, and J. H. Benjamin Brown. Running on four legs as though they were one. IEEE Journal of Robotics and Automation, 2(2):70--82, 1986.
• 26: Joseph Reisinger , Risto Miikkulainen, Acquiring evolvability through adaptive representations, Proceedings of the 9th annual conference on Genetic and evolutionary computation, July 07-11, 2007, London, England [doi>10.1145/1276958.1277164]
• 27: S. V. Shastri. A biologically consistent model of legged locomotion gaits. Biol. Cyber., 76(6):429--440, 1997.
• 28: K. Sims. Evolving 3D morphology and behavior by competition. In Proceedings of Alife IV, pages 28--39, 1994.
• 29: K. O. Stanley, B. D. Bryant, and R. Miikkulainen. Real-time neuroevolution in the NERO video game. IEEE Transactions on Evolutionary Computation, 9(6):653--668, 2005.
• 30: R. A. Téllez, C. Angulo, and D. E. Pardo. Evolving the walking behaviour of a 12 dof quadruped using a distributed neural architecture. In Biologically Inspired Approaches to Advanced Information Technology, pages 5--19, 2006.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
          Subjects: Connectionism and neural nets

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.9 Robotics
          Subjects: Propelling mechanisms

General Terms:
Design, Experimentation, Performance

Keywords:
controllers, coupled cell systems, locomotion, modular neuroevolution, multilegged robots