Automatic configuration of internet services

ABSTRACT

Recent research has found that operators frequently misconfigure Internet services, causing various availability and performance problems. In this paper, we propose a software infrastructure that eliminates several types of misconfiguration by automating the generation of configuration files in Internet services, even as the services evolve. The infrastructure comprises a custom scripting language, configuration file templates, communicating runtime monitors, and heuristic algorithms to detect dependencies between configuration parameters and select ideal configurations. To demonstrate our infrastructure experimentally, we apply it to a realistic online auction service. Our results show that the infrastructure can simplify operation significantly while eliminating 58% of the misconfigurations found in a previous study of the same service. Furthermore, our results show that the infrastructure can efficiently determine the configuration parameters that lead to high performance as the service evolves through a hardware upgrade and the scheduled maintenance of a few nodes.

REFERENCES

• 1: Paul Anderson , Patrick Goldsack , Jim Paterson, SmartFrog Meets LCFG: Autonomous Reconfiguration with Central Policy Control, Proceedings of the 17th USENIX conference on System administration, October 26-31, 2003, San Diego, CA
• 2: Anderson, P., And Scobie, A. LCFG: The Next Generation. In Proceedings of the UKUUG Winter Conference (2002).
• 3: Breiman, L., Friedman, J. H., Olshen, R. A., and Stone, C. J. Classfication and Regression Trees. 1984.
• 4: Brown, A. B., Keller, A., And Hellerstein, J. L. A Model of Configuration Complexity and its Application to a Change Management System. In Proceedings of the 9th IFIP/IEEE International Symposium on Integrated Network Management (2005).
• 5: Burgess, M. Cfengine: A site configuration engine. USENIX Computing systems 8, 3 (1995).
• 6: Ramesh Chandra , Nickolai Zeldovich , Constantine Sapuntzakis , Monica S. Lam, The collective: a cache-based system management architecture, Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation, p.259-272, May 02-04, 2005
• 7: I-Hsin Chung , Jeffrey K. Hollingsworth, Automated Cluster-Based Web Service Performance Tuning, Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing, p.36-44, June 04-06, 2004 [doi>10.1109/HPDC.2004.4]
• 8: Y. Diao , J. L. Hellerstein , S. Parekh , J. P. Bigus, Managing Web server performance with AutoTune agents, IBM Systems Journal, v.42 n.1, p.136-149, January 2003
• 9: Emre Kycyman, Discovering Correctness Constraints for Self-Management of System Configuration, Proceedings of the First International Conference on Autonomic Computing (ICAC'04), p.28-35, May 17-18, 2004
• 10: Kirkpatrick, S., Gelatt, C. D., and Vecchi, M. P. Optimization by Simulated Annealing. Science 220, 4598 (1983).
• 11: Levanta. http://www.levanta.com.
• 12: Kiran Nagaraja , Gustavo Gama , Ricardo Bianchini , Richard P. Martin , Wagner Meira Jr. , Thu D. Nguyen, Quantifying the Performability of Cluster-Based Services, IEEE Transactions on Parallel and Distributed Systems, v.16 n.5, p.456-467, May 2005 [doi>10.1109/TPDS.2005.61]
• 13: Kiran Nagaraja , Fábio Oliveira , Ricardo Bianchini , Richard P. Martin , Thu D. Nguyen, Understanding and dealing with operator mistakes in internet services, Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation, p.5-5, December 06-08, 2004, San Francisco, CA
• 14: Nelder, J. A., and Mead, R. A Simplex Method for Function Minimization. Computer Journal 7, 4 (1965).
• 15: Oliveira, F., Patel, J., Hensbergen, E. V., Gheith, A., and Rajamony, R. Blutopia: Cluster Life-Cycle Management. Tech. Rep. RC23784, IBM Austin, 2005.
• 16: David Oppenheimer , Archana Ganapathi , David A. Patterson, Why do internet services fail, and what can be done about it?, Proceedings of the 4th conference on USENIX Symposium on Internet Technologies and Systems, p.1-1, March 26-28, 2003, Seattle, WA
• 17: David Oppenheimer , David A. Patterson, Architecture and Dependability of Large-Scale Internet Services, IEEE Internet Computing, v.6 n.5, p.41-49, September 2002 [doi>10.1109/MIC.2002.1036037]
• 18: Rice University. DynaServer Project. http://www.cs.rice.edu/CS/Systems/DynaServer, 2003.
• 19: Christopher Stewart , Kai Shen, Performance modeling and system management for multi-component online services, Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation, p.71-84, May 02-04, 2005
• 20: Helen J. Wang , John C. Platt , Yu Chen , Ruyun Zhang , Yi-Min Wang, Automatic misconfiguration troubleshooting with peerpressure, Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation, p.17-17, December 06-08, 2004, San Francisco, CA
• 21: Yi-Min Wang , Chad Verbowski , John Dunagan , Yu Chen , Helen J. Wang , Chun Yuan , Zheng Zhang, STRIDER: A Black-box, State-based Approach to Change and Configuration Management and Support, Proceedings of the 17th USENIX conference on System administration, October 26-31, 2003, San Diego, CA
• 22: Andrew Whitaker , Richard S. Cox , Steven D. Gribble, Configuration debugging as search: finding the needle in the haystack, Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation, p.6-6, December 06-08, 2004, San Francisco, CA

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS

General Terms:
Algorithms, Experimentation, Performance

Keywords:
configuration, internet services, manageability, operator mistakes