General methodology 1: optimising discrete event simulation models using a reinforcement learning agent

ABSTRACT

A reinforcement learning agent has been developed to determine optimal operating policies in a multi-part serial line. The agent interacts with a discrete event simulation model of a stochastic production facility. This study identifies issues important to the simulation developer who wishes to optimise a complex simulation or develop a robust operating policy. Critical parameters pertinent to 'tuning' an agent quickly and enabling it to rapidly learn the system were investigated.

REFERENCES

• 1: Ashkin, R. G., and C. R. Standridge. 1993. Modelling and analysis of Manufacturing Systems Cambridge, MA: The MIT Press.
• 2: Aydin, M. E., and E. Öztemel. 2000. Dynamic jobshop scheduling using reinforcement learning agents. Robotics and Autonomous Systems 33: 169--178.
• 3: Campbell, P., P. Hodgson, and M. Cardew-Hall. 2001. Optimisation of Production Decisions with Stochastic Process Failures. Proceedings of the 16th International Conference On Production Research.
• 4: Das, T. K., and S. Sarkar. 1999. Optimal preventitive maintenance in a production inventory system. IIE Transactions 31: 537--551.
• 5: Harp, S. A., and T. Samad. 2000. Adaptive agents and artificial life: Insights for the power industry. Soft Computing and Intelligent Systems: Theory and Applications. eds. Sinha, N. K., and M. G. Madan, San Diego: Academic Press.
• 6: Jeong, K. 2000. Conceptual frame for development of optimized simulation-based scheduling systems. Expert Systems with Applications 18: 299--306.
• 7: Kaelbling, L. P., M. L. Littman, and A W. Moore. 1996. Reinforcement learning: a survey Journal of Artificial Intelligence Research 4: 237--285.
• 8: Sridhar Mahadevan, Average reward reinforcement learning: foundations, algorithms, and empirical results, Machine Learning, v.22 n.1-3, p.159-195, Jan./Feb./March 1996 [doi>10.1007/BF00114727]
• 9: Mahadevan, S., N. Marchalleck, T. K. Das, and A. Gosavi. 1997. Self-improving factory simulation using continuous-time average-reward reinforcement learning. Proceedings of the 14th International Conference on Machine Learning 202--210.
• 10: Sridhar Mahadevan , Georgios Theocharous, Optimizing Production Manufacturing Using Reinforcement Learning, Proceedings of the Eleventh International Florida Artificial Intelligence Research Society Conference, p.372-377, May 18-20, 1998
• 11: Paternina-Arboleda, C. D., and T. K. Das. 2001. Intelligent dynamic control policies for serial production lines. IIE Transactions 33: 65--77.
• 12: Simone C. Riedmiller , Martin A. Riedmiller, A Neural Reinforcement Learning Approach to Learn Local Dispatching Policies in Production Scheduling, Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, p.764-771, July 31-August 06, 1999
• 13: Jeff G. Schneider , Justin A. Boyan , Andrew W. Moore, Value Function Based Production Scheduling, Proceedings of the Fifteenth International Conference on Machine Learning, p.522-530, July 24-27, 1998
• 14: Richard S. Sutton , Andrew G. Barto, Introduction to Reinforcement Learning, MIT Press, Cambridge, MA, 1998
• 15: Zhang, W., and T. G. Dietterich. 1995. A reinforcement learning approach to job-shop scheduling. Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence. 1114--1120.