Boosting the performance of computing systems through adaptive configuration tuning

ABSTRACT

Good system performance depends on the correct setting of its configuration parameters. It is observed that such optimal configuration relies on the incoming workload of the system. In this paper, we utilize the Markov decision process (MDP) theory and present a reinforcement learning strategy to discover the complex relationship between the system workload and the corresponding optimal configuration. Considering the limitations of current reinforcement learning algorithms used in system management, we present a different learning architecture to facilitate the configuration tuning task which includes two units: the actor and critic. While the actor realizes a stochastic policy that maps the system state to the corresponding configuration setting, the critic uses a value function to provide the reinforcement feedback to the actor. Both the actor and critic are implemented by multiple layer neural networks, and the error back-propagation algorithm is used to adjust the network weights based on the temporal difference error produced in the learning. Experimental results demonstrate that the proposed learning process can identify the correct configuration tuning rule which in turn improves the system performance significantly.

REFERENCES

• 1: Mohamed N. Bennani , Daniel A. Menasce, Resource Allocation for Autonomic Data Centers using Analytic Performance Models, Proceedings of the Second International Conference on Automatic Computing, p.229-240, June 13-16, 2005 [doi>10.1109/ICAC.2005.50]
• 2: Simon Haykin, Neural Networks: A Comprehensive Foundation, Prentice Hall PTR, Upper Saddle River, NJ, 1998
• 3: Chenyang Lu , Ying Lu , Tarek F. Abdelzaher , John A. Stankovic , Sang Hyuk Son, Feedback Control Architecture and Design Methodology for Service Delay Guarantees in Web Servers, IEEE Transactions on Parallel and Distributed Systems, v.17 n.9, p.1014-1027, September 2006 [doi>10.1109/TPDS.2006.123]
• 4: Daniel A. Menasce , Lawrence W. Dowdy , Virgilio A. F. Almeida, Performance by Design: Computer Capacity Planning By Example, Prentice Hall PTR, Upper Saddle River, NJ, 2004
• 5: Anooshiravan Saboori , Guofei Jiang , Haifeng Chen, Autotuning Configurations in Distributed Systems for Performance Improvements Using Evolutionary Strategies, Proceedings of the 2008 The 28th International Conference on Distributed Computing Systems, p.769-776, June 17-20, 2008 [doi>10.1109/ICDCS.2008.11]
• 6: Richard S. Sutton , Andrew G. Barto, Introduction to Reinforcement Learning, MIT Press, Cambridge, MA, 1998
• 7: C. Tapus, I. Chung, and J. Hollingsworth. Active harmony: Towards automated performance tuning. In Proceedings of High Performance Networking and Computing (SC '03), Baltimore, USA, 2003.
• 8: G. Tesauro. Online resource allocation using decompositional reinforcement learning. In The Twentieth National Conference on Artificial Intelligence and the Seventeenth Innovative Applications of Artificial Intelligence Conference (AAAI), pages 886--891, 2005.
• 9: David Vengerov, A reinforcement learning framework for online data migration in hierarchical storage systems, The Journal of Supercomputing, v.43 n.1, p.1-19, January 2008 [doi>10.1007/s11227-007-0135-3]
• 10: Christopher J. C. H. Watkins , Peter Dayan, Technical Note: $\backslash cal\; Q$-Learning, Machine Learning, v.8 n.3-4, p.279-292, May 1992 [doi>10.1007/BF00992698]
• 11: Bowei Xi , Zhen Liu , Mukund Raghavachari , Cathy H. Xia , Li Zhang, A smart hill-climbing algorithm for application server configuration, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988711]
• 12: Tao Ye , Shivkumar Kalyanaraman, A recursive random search algorithm for large-scale network parameter configuration, Proceedings of the 2003 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, June 11-14, 2003, San Diego, CA, USA
• 13: Wei Zheng , Ricardo Bianchini , Thu D. Nguyen, Automatic configuration of internet services, Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007, March 21-23, 2007, Lisbon, Portugal

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.4 System Management

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning

General Terms:
Algorithms, Management

Keywords:
configuration tuning, reinforcement learning, system management