Learning trust strategies in reputation exchange networks

ABSTRACT

An agent's trust decision strategy consists of the agent's policies for making trust-related decisions, such as who to trust, how trustworthy to be, what reputations to believe, and when to tell truthful reputations. In reputation exchange networks, learning trust decision strategies is complex, compared to non-reputation-communicating systems. When potential partners may exchange reputation information about an agent, the agent's interactions with one partner are no longer independent from interactions with another; partners may tell each other about their experiences with the agent, influencing future behavior. This research enumerates the types of decisions an agent faces in reputation exchange networks, explains the interdependencies between these decisions, and correlates rewards to each decision. Experimental results using the Agent Reputation and Trust (ART) Testbed demonstrate the success of strategy-learning agents over agents employing naive strategies. The variation in performance of reputation-based learning vs. experience-based learning over different opponents illustrates the need to dynamically determine when to utilize reputations vs. experience in making trust decisions.

REFERENCES

• 1: ART Testbed Team. Agent Reputation and Trust Testbed. http://www.lips.utexas.edu/art-testbed/, 2005.
• 2: Axelrod, R. The Evolution of Cooperation. New York: Basic Books, 1984.
• 3: Rajatish Mukherjee , Bikramjit Banerjee , Sandip Sen, Learning Mutual Trust, Proceedings of the workshop on Deception, Fraud, and Trust in Agent Societies held during the Autonomous Agents Conference: Trust in Cyber-societies, Integrating the Human and Artificial Perspectives, p.145-158, June 01, 2000
• 4: Banerjee, B. and J. Peng. "Countering Deception in Multiagent Reinforcement Learning," Proceedings of the Workshop on Trust, Privacy, Deception and Fraud in Agent Societies at AAMAS-03, Melbourne, Australia, pp. 1--5, 2003.
• 5: K. Suzanne Barber , Joonoo Kim, Belief Revision Process Based on Trust: Agents Evaluating Reputation of Information Sources, Proceedings of the workshop on Deception, Fraud, and Trust in Agent Societies held during the Autonomous Agents Conference: Trust in Cyber-societies, Integrating the Human and Artificial Perspectives, p.73-82, June 01, 2000
• 6: Barber, K. S. and J. Kim. "Soft Security: Isolating Unreliable Agents," Proceedings of The Workshop on Deception, Fraud, and Trust in Agent Societies at AAMAS-02, Bologna, Italy, pp. 8--17, 2002.
• 7: Barber, K. S., K. Fullam, and J. Kim. "Challenges for Trust, Fraud, and Deception Research in Multi-agent Systems," Trust, Reputation, and Security: Theories and Practice, R. Falcone, K. S. Barber, L. Korba and M. Singh, Eds., Springer: pp. 8--14, 2003.
• 8: Barber, K. S. and K. Fullam. "Applying Reputation Models to Continuous Belief Revision," Proceedings of The Workshop on Deception, Fraud and Trust in Agent Societies at AAMAS-03, Melbourne, Australia, pp. 6--15, 2003.
• 9: Crandall, J. W. and M. A. Goodrich. "Establishing Reputation Using Social Commitment in Repeated Games," Proceedings of the Workshop on Learning and Evolution in Agent Based Systems at AAMAS-04, New York, pp. 12--17, 2004.
• 10: Karen K. Fullam , Tomas B. Klos , Guillaume Muller , Jordi Sabater , Andreas Schlosser , Zvi Topol , K. Suzanne Barber , Jeffrey S. Rosenschein , Laurent Vercouter , Marco Voss, A specification of the Agent Reputation and Trust (ART) testbed: experimentation and competition for trust in agent societies, Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems, July 25-29, 2005, The Netherlands [doi>10.1145/1082473.1082551]
• 11: Huynh, T. D., N. R. Jennings, and N. Shadbolt. "On Handling Inaccurate Witness Reports," Proceedings of 8th International Workshop on Trust in Agent Societies at AAMAS-05, Utrecht, pp. 63--77, 2005.
• 12: Catholijn M. Jonker , Jan Treur, Formal Analysis of Models for the Dynamics of Trust Based on Experiences, Proceedings of the 9th European Workshop on Modelling Autonomous Agents in a Multi-Agent World: MultiAgent System Engineering, p.221-231, June 30-July 02, 1999
• 13: Littman, M. and P. Stone. "Leading Best-Response Strategies in Repeated Games," Proceedings of the Workshop on Economic Agents, Models, and Mechanisms, at IJCAI-01, Seattle, Washington, 2001.
• 14: Lik Mui , Mojdeh Mohtashemi , Ari Halberstadt, Notions of reputation in multi-agents systems: a review, Proceedings of the first international joint conference on Autonomous agents and multiagent systems: part 1, July 15-19, 2002, Bologna, Italy [doi>10.1145/544741.544807]
• 15: Tuomas Sandholm , Robert H. Crites, On Multiagent Q-Learning in a Semi-Competitive Domain, Proceedings of the Workshop on Adaption and Learning in Multi-Agent Systems, p.191-205, August 21, 1995
• 16: Schillo, M., P. Funk, and M. Rovatsos. "Using Trust for Detecting Deceitful Agents in Artificial Societies," Proceedings of The Applied Artificial Intelligence Journal, Special Issue on Deception, Fraud and Trust in Agent Societies, pp. 825--848, 2000.
• 17: Shi, J., G. Bochmann, and C. Adams. "Dealing with Recommendations in a Statistical Trust Model," Proceedings of the Workshop on Trust in Agent Societies at AAMAS-05, Utrecht, pp. 144--155, 2005.
• 18: W. T. Luke Teacy , Jigar Patel , Nicholas R. Jennings , Michael Luck, Coping with inaccurate reputation sources: experimental analysis of a probabilistic trust model, Proceedings of the fourth international joint conference on Autonomous agents and multiagent systems, July 25-29, 2005, The Netherlands [doi>10.1145/1082473.1082624]
• 19: Michael Weinberg , Jeffrey S. Rosenschein, Best-Response Multiagent Learning in Non-Stationary Environments, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, p.506-513, July 19-23, 2004, New York, New York [doi>10.1109/AAMAS.2004.75]
• 20: Yolum, P. and M. P. Singh. "Self-Organizing Referral Networks: A Process View of Trust and Authority," Engineering Self-organizing Systems, Lecture Notes in Artificial Intelligence, pp. 195--211, 2003.
• 21: Bin Yu , Munindar P. Singh, An evidential model of distributed reputation management, Proceedings of the first international joint conference on Autonomous agents and multiagent systems: part 1, July 15-19, 2002, Bologna, Italy [doi>10.1145/544741.544809]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.11 Distributed Artificial Intelligence
            Subjects: Intelligent agents

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.11 Distributed Artificial Intelligence
            Subjects: Multiagent systems

General Terms:
Algorithms, Experimentation

Keywords:
ART testbed, multi-agent systems, reputation, trust