Learning complex robot control using evolutionary behavior based systems

ABSTRACT

Evolving a monolithic solution for complex robotic problems is hard. One of the reasons for this is the difficulty of defining a global fitness function that leads to a solution with desired operating properties. The problem with a global fitness function is that it may not reward intermediate solutions that would ultimately lead to the desired operating properties. A possible way to solve such a problem is to decompose the solution space into smaller subsolutions with lower number of intrinsic dimensions. In this paper, we apply the design principles of behavior based systems to decompose a complex robot control task into subsolutions and show how to incrementally modify the fitness function that (1) results in desired operating properties as the subsolutions are learned, and (2) avoids the need to learn the coordination of behaviors separately. We demonstrate our method by learning to control a quadrocopter flying vehicle.

REFERENCES

• 1: E. Altug, J. P. Ostrowski, and C. J. Taylor. Quadrotor control using dual camera visual feedback. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2003), pages 4294--4299, 2003.
• 2: Ronald C. Arkin, An Behavior-based Robotics, MIT Press, Cambridge, MA, 1998
• 3: J. Bongard. Behavior chaining: Incremental behavior integration for evolutionary robotics. Artificial Life, 2008.
• 4: M. Botvinick, Y. Niv, and A. C. Barto. Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective. Cognition, 2008. in press.
• 5: S. Bouabdallah and R. Siegwart. Full control of a quadrotor. In Proceedings of The IEEE International Conference on Intelligent Robots (IROS), pages 153--158, 2007.
• 6: R. A. Brooks. A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation, 2(1):14--23, 1986.
• 7: Rodney A. Brooks, A Robot That Walks; Emergent Behaviors from a Carefully Evolved Network, Massachusetts Institute of Technology, Cambridge, MA, 1989
• 8: Raffaele Calabretta , Stefano Nolfi , Domenico Parisi , Günter P. Wagner, Duplication of modules facilitates the evolution of functional specialization, Artificial Life, v.6 n.1, p.69-84, Dec. 1999 [doi>10.1162/106454600568320]
• 9: Takashi Gomi , Ann Griffith, Developing Intelligent Wheelchairs for the Handicapped, Assistive Technology and Artificial Intelligence, Applications in Robotics, User Interfaces and Natural Language Processing, p.150-178, January 1998
• 10: J. E. Gray and W. Murray. A derivation of an analytic expression for the tracking index for alpha-beta-gamma filter. IEEE Transactions on Aerospace and Electronic Systems, 29(3):1064--1065, 1993.
• 11: Frederic Gruau , Kameel Quatramaran, Cellular encoding for interactive evolutionary robotics, CWI (Centre for Mathematics and Computer Science), Amsterdam, The Netherlands, 1996
• 12: T. Hamel, R. Mahony, R. Lozano, and J. Ostrowski. Dynamic modelling and configuration stabilization for an X4-flyer. In International Federation of Automatic Control Symposium, IFAC, 2002.
• 13: Nikolaus Hansen , Andreas Ostermeier, Completely Derandomized Self-Adaptation in Evolution Strategies, Evolutionary Computation, v.9 n.2, p.159-195, June 2001 [doi>10.1162/106365601750190398]
• 14: P. R. Kalata. Alpha-beta target tracking systems: A survey. In American Control Conference, pages 832--836, 1992.
• 15: Yohannes Kassahun , Jose de Gea , Mark Edgington , Jan Hendrik Metzen , Frank Kirchner, Accelerating neuroevolutionary methods using a Kalman filter, Proceedings of the 10th annual conference on Genetic and evolutionary computation, July 12-16, 2008, Atlanta, GA, USA [doi>10.1145/1389095.1389365]
• 16: Yohannes Kassahun , Jose Gea , Jan Hendrik Metzen , Mark Edgington , Frank Kirchner, EANT+KALMAN: An Efficient Reinforcement Learning Method for Continuous State Partially Observable Domains, Proceedings of the 31st annual German conference on Advances in Artificial Intelligence, September 23-26, 2008, Kaiserslautern, Germany [doi>10.1007/978-3-540-85845-4_30]
• 17: F. Kirchner. Q-learning of complex behaviors on a six-legged walking machine. Journal of Robotics and Autonomous Systems, 25:256--263, 1998.
• 18: Wei-Po Lee , John Hallam , Henrik Hautop Lund, Learning Complex Robot Behaviours by Evolutionary Computing with Task Decomposition, Proceedings of the 6th European Workshop on Learning Robots, p.155-172, August 01-02, 1997
• 19: Jakob Schwendner, Homography Based State Estimation for Aerial Robots, Proceedings of the 31st annual German conference on Advances in Artificial Intelligence, September 23-26, 2008, Kaiserslautern, Germany [doi>10.1007/978-3-540-85845-4_41]
• 20: T. Ziemke, N. Bergfeldt, G. Buason, T. Susi, and H. Svensson. Evolving cognitive scaffolding and environment adaptation: a new research direction for evolutionary robotics. Connection Science, 16(4):339--350, 2004.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.9 Robotics

General Terms:
Algorithms

Keywords:
behavior based robotics, kalman filter, neuroevolution