Artificial agents learning human fairness

ABSTRACT

Recent advances in technology allow multi-agent systems to be deployed in cooperation with or as a service for humans. Typically, those systems are designed assuming individually rational agents, according to the principles of classical game theory. However, research in the field of behavioral economics has shown that humans are not purely self-interested: they strongly care about fairness. Therefore, multi-agent systems that fail to take fairness into account, may not be sufficiently aligned with human expectations and may not reach intended goals. In this paper, we present a computational model for achieving fairness in adaptive multi-agent systems. The model uses a combination of Continuous Action Learning Automata and the Homo Egualis utility function. The novel contribution of our work is that this function is used in an explicit, computational manner. We show that results obtained by agents using this model are compatible with experimental and analytical results on human fairness, obtained in the field of behavioral economics.

REFERENCES

• 1: H. Aldewereld. Autonomy vs. Conformity: an Institutional Perspective on Norms and Protocols. PhD thesis, Universiteit Utrecht, 2007.
• 2: K. Basu. The Traveler's Dilemma. Scientific American, Volume 296, Number 6:68--73, 2007.
• 3: K. Binmore. Natural Justice. Oxford University Press, 2005.
• 4: S. Bowles, R. Boyd, E. Fehr, and H. Gintis. Homo reciprocans: A Research Initiative on the Origins, Dimensions, and Policy Implications of Reciprocal Fairness. Advances in Complex Systems, 4:1--30, 1997.
• 5: Multiagent learning using a variable learning rate, Artificial Intelligence, v.136 n.2, p.215-250, April 2002 [doi>10.1016/S0004-3702(02)00121-2]
• 6: G. Charness and M. Rabin. Understanding Social Preferences with Simple Tests. Quarterly Journal of Economics, 117:817--869, 2002.
• 7: Y. Chevaleyre, P. E. Dunne, U. Endriss, J. Lang, M. Lemaître, N. Maudet, J. Padget, S. Phelps, J. A. Rodriguez-Aguilar, and P. Sousa. Issues in Multiagent Resource Allocation. Informatica, 30:3--31, 2006.
• 8: Yann Chevaleyre , Ulle Endriss , Jérôme Lang , Nicolas Maudet, A Short Introduction to Computational Social Choice, Proceedings of the 33rd conference on Current Trends in Theory and Practice of Computer Science, January 20-26, 2007, Harrachov, Czech Republic [doi>10.1007/978-3-540-69507-3_4]
• 9: A. Dannenberg, T. Riechmann, B. Sturm, and C. Vogt. Inequity Aversion and Individual Behavior in Public Good Games: An Experimental Investigation. SSRN eLibrary, 2007.
• 10: Steven de Jong , Karl Tuyls , Ida Sprinkhuizen-Kuyper, Robust and Scalable Coordination of Potential-Field Driven Agents, Proceedings of the International Conference on Computational Inteligence for Modelling Control and Automation and International Conference on Intelligent Agents Web Technologies and International Commerce, p.230, November 28-December 01, 2006 [doi>10.1109/CIMCA.2006.191]
• 11: S. de Jong, K. Tuyls, K. Verbeeck, and N. Roos. Priority awareness: towards a computational model of human fairness for multi-agent systems. Adaptive Agents and Multi-Agent Systems III - Lecture Notes in Artificial Intelligence, 4865, 2008.
• 12: I. Erev and A. E. Roth. Predicting how people play games with unique, mixed strategy equilibria. American Economic Review, 88:848--881, 1998.
• 13: A. Falk and U. Fischbacher. A theory of reciprocity. Games and Economic Behavior, 54:293--315, 2006.
• 14: E. Fehr. Don't lose your reputation. Nature, 432:499--500, 2004.
• 15: E. Fehr and S. Gaechter. Fairness and Retaliation: The Economics of Reciprocity. Journal of Economic Perspectives, 14:159--181, 2000.
• 16: E. Fehr and S. Gaechter. Altruistic punishment in humans. Nature, 415:137--140, 2002.
• 17: E. Fehr and K. Schmidt. A Theory of Fairness, Competition and Cooperation. Quarterly Journal of Economics, 114:817--868, 1999.
• 18: H. Gintis. Game Theory Evolving: A Problem-Centered Introduction to Modeling Strategic Interaction. Princeton University Press, 2001.
• 19: W. Gueth, R. Schmittberger, and B. Schwarze. An Experimental Analysis of Ultimatum Bargaining. Journal of Economic Behavior and Organization, 3 (4):367--388, 1982.
• 20: C. Hauert, S. D. Monte, J. Hofbauer, and K. Sigmund. Volunteering as red queen mechanism for cooperation in public goods games. Science, 296:1129--1132, 2002.
• 21: C. Hauert, A. Traulsen, H. Brandt, M. Nowak, and K. Sigmund. Via freedom to coercion: the emergence of costly punishment. Science, 316:1905--1907, 2007.
• 22: J. Henrich, R. Boyd, S. Bowles, C. Camerer, E. Fehr, and H. Gintis. Foundations of Human Sociality: Economic Experiments and Ethnographic Evidence from Fifteen Small-Scale Societies. Oxford University Press, 2004.
• 23: M. Jackson. Mechanism Theory. Humanities and Social Sciences, October:228--277, 2000.
• 24: X. Mao, A. ter Mors, N. Roos, and C. Witteveen. Agent-Based Scheduling for Aircraft Deicing. In P.-Y. Schobbens, W. Vanhoof, and G. Schwanen, editors, Proceedings of the 18th Belgium - Netherlands Conference on Artificial Intelligence, pages 229--236. BNVKI, October 2006.
• 25: M. Milinski, D. Semmann, and H. Krambeck. Reputation helps solve the tragedy of the commons. Nature, 415:424--426, 2002.
• 26: Kumpati S. Narendra , Mandayam A. L. Thathachar, Learning automata: an introduction, Prentice-Hall, Inc., Upper Saddle River, NJ, 1989
• 27: J. Nash. Equilibrium Points in N-person Games. Proceedings of the National Academy of Sciences, 36:48--49, 1950.
• 28: J. Nash. The Bargaining Problem. Econometrica, 18:155--162, 1950.
• 29: M. Nowak, K. Page, and K. Sigmund. Fairness versus reason in the Ultimatum Game. Science, 289:1773--1775, 2000.
• 30: R. Nydegger and H. Owen. Two-person bargaining, an experimental test of the Nash axioms. International Journal of Game Theory, 3:239--250, 1974.
• 31: H. Oosterbeek, R. Sloof, and G. van de Kuilen. Cultural Differences in Ultimatum Game Experiments: Evidence from a Meta-Analysis. Experimental Economics, 7:171--188, 2004.
• 32: K. Panchanathan and R. Boyd. Indirect reciprocity can stabilize cooperation without the second-order free rider problem. Nature, 432:499--502, 2004.
• 33: D. C. Parkes. Computational Mechanism Design. In Lecture notes of Tutorials at 10th Conf. on Theoretical Aspectsof Rationality and Knowledge (TARK-05). Institute of Mathematical Sciences, University of Singapore, 2008.
• 34: J. A. Rodriguez-Aguilar. On the design and construction of agent-mediated electronic institutions. PhD thesis, Monografies de l'Institut d'Investigació en Intelligència Artificial, 2003.
• 35: A. Roth and M. Malouf. Game Theoretic Models and the Role of Information in Bargaining. Psychological Review, 86:574--594, 1979.
• 36: Stuart J. Russell , Peter Norvig, Artificial intelligence: a modern approach, Prentice-Hall, Inc., Upper Saddle River, NJ, 1995
• 37: F. Santos, J. Pacheco, and T. Lenaerts. Cooperation Prevails When Individuals Adjust Their Social Ties. PLoS Comput. Biol., 2(10):1284--1291, 2006.
• 38: F. Santos, J. Pacheco, and T. Lenaerts. Evolutionary Dynamics of Social Dilemmas in Structured Heterogeneous Populations. Proc. Natl. Acad. Sci. USA, 103:3490--3494, 2006.
• 39: S. Sen and S. Airiau. Emergence of norms through social learning. In Proceedings of the Twentieth International Joint Conference on Artificial Intelligence, pages 1507--1512, 2007.
• 40: Yoav Shoham , Rob Powers , Trond Grenager, If multi-agent learning is the answer, what is the question?, Artificial Intelligence, v.171 n.7, p.365-377, May, 2007 [doi>10.1016/j.artint.2006.02.006]
• 41: K. Sigmund, C. Hauert, and M. Nowak. Reward and punishment. Proceedings of the National Academy of Sciences, 98(19):10757--10762, 2001.
• 42: H. Simon. Models of Man. John Wiley, 1957.
• 43: M. A. L. Thathachar , P. S. Sastry, Networks of Learning Automata: Techniques for Online Stochastic Optimization, Springer-Verlag New York, Inc., Secaucus, NJ, 2003
• 44: Karl Tuyls , Ann Nowé, Evolutionary game theory and multi-agent reinforcement learning, The Knowledge Engineering Review, v.20 n.1, p.63-90, March 2005 [doi>10.1017/S026988890500041X]
• 45: J. Vazquez-Salceda, H. Aldewereld, and F. Dignum. Norms in multiagent systems: from theory to practice. Comput. Syst. Sci. Eng., 20(4), 2005.
• 46: Katja Verbeeck , Ann Nowé , Johan Parent , Karl Tuyls, Exploring selfish reinforcement learning in repeated games with stochastic rewards, Autonomous Agents and Multi-Agent Systems, v.14 n.3, p.239-269, June 2007 [doi>10.1007/s10458-006-9007-0]
• 47: T. Yamagishi. The provision of a sanctioning system as a public good. J. Person. and Soc. Psych., 51(1):110--116, 1986.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.11 Distributed Artificial Intelligence

  J. Computer Applications
  J.4 SOCIAL AND BEHAVIORAL SCIENCES

General Terms:
Algorithms, Design, Human Factors

Keywords:
fairness, homo egualis, reinforcement learning