Privacy-preserving reinforcement learning

ABSTRACT

We consider the problem of distributed reinforcement learning (DRL) from private perceptions. In our setting, agents' perceptions, such as states, rewards, and actions, are not only distributed but also should be kept private. Conventional DRL algorithms can handle multiple agents, but do not necessarily guarantee privacy preservation and may not guarantee optimality. In this work, we design cryptographic solutions that achieve optimal policies without requiring the agents to share their private information.

REFERENCES

• 1: Naoki Abe , Naval Verma , Chid Apte , Robert Schroko, Cross channel optimized marketing by reinforcement learning, Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, August 22-25, 2004, Seattle, WA, USA [doi>10.1145/1014052.1016912]
• 2: Cogill, R., Rotkowitz, M., Van Roy, B., & Lall, S. (2006). An Approximate Dynamic Programming Approach to Decentralized Control of Stochastic Systems. LNCIS, 329, 243--256.
• 3: Dåmgard, I., & Jurik, M. (2001). A Generalisation, a Simplification and Some Applications of Paillier's Probabilistic Public-Key System. Public Key Cryptography 2001. Springer.
• 4: Sébastien Gambs , Balázs Kégl , Esma Aïmeur, Privacy-preserving boosting, Data Mining and Knowledge Discovery, v.14 n.1, p.131-170, February 2007 [doi>10.1007/s10618-006-0051-9]
• 5: Oded Goldreich, Foundations of Cryptography: Volume 2, Basic Applications, Cambridge University Press, New York, NY, 2004
• 6: Geetha Jagannathan , Rebecca N. Wright, Privacy-preserving distributed k-means clustering over arbitrarily partitioned data, Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, August 21-24, 2005, Chicago, Illinois, USA [doi>10.1145/1081870.1081942]
• 7: Kearns, M., Tan, J., & Wortman, J. (2007). Privacy-Preserving Belief Propagation and Sampling. NIPS 20.
• 8: Lindell, Y., & Pinkas, B. (2002). Privacy Preserving Data Mining. Journal of Cryptology, 15, 177--206.
• 9: Dahlia Malkhi , Noam Nisan , Benny Pinkas , Yaron Sella, Fairplay—a secure two-party computation system, Proceedings of the 13th conference on USENIX Security Symposium, p.20-20, August 09-13, 2004, San Diego, CA
• 10: Moallemi, C. C., & Roy, B. V. (2004). Distributed optimization in adaptive networks. NIPS 16.
• 11: Sakuma, J., & Kobayashi, S. (2008). Large-scale kmeans Clustering with User-Centric Privacy Preservation. Pacific-Asia Conf. on Knowledge Discovery and Data Mining (PAKDD) 2008, to appear.
• 12: Jeff G. Schneider , Weng-Keen Wong , Andrew W. Moore , Martin A. Riedmiller, Distributed Value Functions, Proceedings of the Sixteenth International Conference on Machine Learning, p.371-378, June 27-30, 1999
• 13: Richard S. Sutton, Reinforcement Learning, Kluwer Academic Publishers, Norwell, MA, 1992
• 14: Watkins, C. (1989). Learning from Delayed Rewards. Cambridge University.
• 15: Andrew Chi-Chih Yao, How to generate and exchange secrets, Proceedings of the 27th Annual Symposium on Foundations of Computer Science (sfcs 1986), p.162-167, October 27-29, 1986 [doi>10.1109/SFCS.1986.25]
• 16: Hwanjo Yu , Xiaoqian Jiang , Jaideep Vaidya, Privacy-preserving SVM using nonlinear kernels on horizontally partitioned data, Proceedings of the 2006 ACM symposium on Applied computing, April 23-27, 2006, Dijon, France [doi>10.1145/1141277.1141415]
• 17: Sheng Zhang , Fillia Makedon, Privacy preserving learning in negotiation, Proceedings of the 2005 ACM symposium on Applied computing, March 13-17, 2005, Santa Fe, New Mexico [doi>10.1145/1066677.1066865]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning

Additional Classification:
  E. Data
  E.3 DATA ENCRYPTION

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.11 Distributed Artificial Intelligence
            Subjects: Intelligent agents

General Terms:
Algorithms, Design, Security, Theory