Learning to cooperate in a continuous tragedy of the commons

ABSTRACT

In previous work, we discussed that social dilemmas are often present in multi-agent systems [3]. Social dilemmas are problems in which we can only find a good solution if we consider the benefit of others in addition to our own benefit. Altruistic punishment has been identified as an important mechanism to enforce this consideration. However, as the punishment is altruistic, deciding whether to punish essentially entails a second-order social dilemma. We developed a methodology that allowed individually learning agents to reach satisfactory solutions in a social dilemma with a continuous strategy space, called the Ultimatum Game [2]. We extended this methodology to thousands of agents, using social networks [4]. Moreover, we devoted attention to the tragedy of the commons, a social dilemma typically exemplified by the Public Goods Game (PGG) [1]. In this game, which is played repeatedly, every agent i (out of n) has to decide on an investment μ_i ε [0, C]. The summed investment is multiplied by a factor 1 < r < n, and equally distributed over all agents. Agent i's individual benefit (or reward) is maximized by μ_i = 0, whereas the group gains the most by collectively playing μ_i = C. Altruistic punishment (i.e., reducing an other agent's reward by an amount e, with a cost c < e to the punisher) allows agents to force others to invest a higher amount, but performing such punishment is clearly not individually rational. In earlier work, we restricted ourselves to a small number of strategies and/or agents in this game [1].

REFERENCES

• 1: S. de Jong and K. Tuyls. Learning to cooperate in public-goods interactions. 2008. Presented at the EUMAS'08 Workshop, Bath, UK, December 18--19.
• 2: Steven de Jong , Karl Tuyls , Katja Verbeeck, Artificial agents learning human fairness, Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems, May 12-16, 2008, Estoril, Portugal
• 3: Steven de Jong , Karl Tuyls , Katja Verbeeck, Fairness in multi-agent systems, The Knowledge Engineering Review, v.23 n.2, p.153-180, June 2008 [doi>10.1017/S026988890800132X]
• 4: S. de Jong, S. Uyttendaele, and K. Tuyls. Learning to Reach Agreement in a Continuous Ultimatum Game. Journal of Artificial Intelligence Research, 33:551--574, 2008.
• 5: E. Fehr and K. Schmidt. A Theory of Fairness, Competition and Cooperation. Quart. J. of Economics, 114:817--868, 1999.
• 6: F. C. Santos, J. M. Pacheco, and T. Lenaerts. Cooperation Prevails When Individuals Adjust Their Social Ties. PLoS Comput. Biol., 2(10): 1284--1291, 2006.
• 7: M. A. L. Thathachar , P. S. Sastry, Networks of Learning Automata: Techniques for Online Stochastic Optimization, Springer-Verlag New York, Inc., Secaucus, NJ, 2003

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.11 Distributed Artificial Intelligence

  J. Computer Applications
  J.4 SOCIAL AND BEHAVIORAL SCIENCES

General Terms:
Algorithms, Design, Human Factors

Keywords:
complex networks, learning, punishment, tragedy of the commons