Team formation and communication restrictions in collectives

ABSTRACT

A collective of agents often needs to maximize a "world utility" function which rates the performance of an entire system, while subject to communication restrictions among the agents. Such communication restrictions make it difficult for agents which try to pursue their own "private" utilities to take actions that also help optimize the world utility. Team formation presents a solution to this problem, where by joining other agents, an agent can significantly increase its knowledge about the environment and improve its chances of both optimizing its own utility and that its doing so will contribute to the world utility. In this article we show how utilities that have been previously shown to be effective in collectives can be modified to be more effective in domains with moderate communication restrictions resulting in performance improvements of up to 75%. Additionally we show that even severe communication constraints can be overcome by forming teams where each agent of a team shares the same utility, increasing performance an additional 25%. We show that utilities and team sizes can be manipulated to form the best compromise between how "aligned" an agent's utility is with the world utility and how easily an agent can learn that utility.

REFERENCES

• 1: J. Fredslund and M. J Mataric. Robots in formation using local information. In Proc. IAS-7, March 2002.
• 2: David V. Pynadath , Milind Tambe , Nicolas Chauvat , Lawrence Cavedon, Toward Team-Oriented Programming, 6th International Workshop on Intelligent Agents VI, Agent Theories, Architectures, and Languages (ATAL),, p.233-247, July 15-17, 1999
• 3: Kagan Tumer , Adrian K. Agogino , David H. Wolpert, Learning sequences of actions in collectives of autonomous agents, Proceedings of the first international joint conference on Autonomous agents and multiagent systems: part 1, July 15-19, 2002, Bologna, Italy [doi>10.1145/544741.544832]
• 4: D. H. Wolpert and K. Tumer. Optimal payoff functions for members of collectives. Advances in Complex Systems, 4(2/3):265--279, 2001.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.11 Distributed Artificial Intelligence
            Subjects: Multiagent systems

General Terms:
Economics, Experimentation, Performance, Theory

Keywords:
MAS, q-learning, reinforcement learning