Learning from actions not taken: a multiagent learning algorithm

ABSTRACT

Learning in multiagent systems is generally slow because the agent has to extract its correct policy through not only through its interaction with the environment, but also from its interactions with other learning agents. In this paper, we present an approach that significantly improves the learning speed in multiagent systems by allowing an agent to up-date its estimate of the rewards for all its available actions, not just the action that was taken. Our results show that the rewards on such "actions not taken" are beneficial early in training, particularly when agent teams are leveraged to estimate those rewards.

REFERENCES

• 1: Adrian K. Agogino , Kagan Tumer, Analyzing and visualizing multiagent rewards in dynamic and stochastic domains, Autonomous Agents and Multi-Agent Systems, v.17 n.2, p.320-338, October 2008 [doi>10.1007/s10458-008-9046-9]
• 2: N. Khani and K. Tumer. Fast Multiagent Learning: Cashing in on Team Knowledge. In Intel. Engr. Systems Though Artificial Neural Nets 18:3--11, 2008.
• 3: Peter Stone, Layered Learning in Multiagent Systems: A Winning Approach to Robotic Soccer, MIT Press, Cambridge, MA, 2000
• 4: Kagan Tumer , Adrian Agogino, Distributed agent-based air traffic flow management, Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems, May 14-18, 2007, Honolulu, Hawaii [doi>10.1145/1329125.1329434]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.11 Distributed Artificial Intelligence

General Terms:
Algorithms, Performance

Keywords:
multiagent learning, multiagent systems