Automated control of multiple virtualized resources

ABSTRACT

Virtualized data centers enable sharing of resources among hosted applications. However, it is difficult to satisfy service-level objectives(SLOs) of applications on shared infrastructure, as application workloads and resource consumption patterns change over time. In this paper, we present AutoControl, a resource control system that automatically adapts to dynamic workload changes to achieve application SLOs. AutoControl is a combination of an online model estimator and a novel multi-input, multi-output (MIMO) resource controller. The model estimator captures the complex relationship between application performance and resource allocations, while the MIMO controller allocates the right amount of multiple virtualized resources to achieve application SLOs. Our experimental evaluation with RUBiS and TPC-W benchmarks along with production-trace-driven workloads indicates that AutoControl can detect and mitigate CPU and disk I/O bottlenecks that occur over time and across multiple nodes by allocating each resource accordingly. We also show that AutoControl can be used to provide service differentiation according to the application priorities during resource contention.

REFERENCES

• 1: Tarek F. Abdelzaher , Kang G. Shin , Nina Bhatti, Performance Guarantees for Web Server End-Systems: A Control-Theoretical Approach, IEEE Transactions on Parallel and Distributed Systems, v.13 n.1, p.80-96, January 2002 [doi>10.1109/71.980028]
• 2: C. Amza, A. Chanda, A.L. Cox, S. Elnikety, R. Gil, K. Rajamani, E. Cecchet, and J. Marguerite. Specification and implementation of dynamic Web site benchmarks. In Proceedings of the 5th IEEE Annual Workshop on Workload Characterization, October 2002.
• 3: Mohit Aron , Peter Druschel , Willy Zwaenepoel, Cluster reserves: a mechanism for resource management in cluster-based network servers, Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.90-101, June 18-21, 2000, Santa Clara, California, United States
• 4: Karl Johan Astrom , Bjorn Wittenmark, Adaptive Control, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1994
• 5: Gaurav Banga , Peter Druschel , Jeffrey C. Mogul, Resource containers: a new facility for resource management in server systems, Proceedings of the third symposium on Operating systems design and implementation, p.45-58, February 1999, New Orleans, Louisiana, United States
• 6: Paul Barham , Boris Dragovic , Keir Fraser , Steven Hand , Tim Harris , Alex Ho , Rolf Neugebauer , Ian Pratt , Andrew Warfield, Xen and the art of virtualization, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 7: Morris Marden, An Architectural Evaluation of Java TPC-W, Proceedings of the 7th International Symposium on High-Performance Computer Architecture, p.229, January 20-24, 2001
• 8: D.D. Chambliss, G.A. Alvarez, P. Pandey, D. Jadav, J. Xu, R. Menon, and T.P. Lee. Performance virtualization for large-scale storage systems. In Proceedings of the 22nd Symposium on Reliable Distributed Systems (SRDS), October 2003.
• 9: Jeffrey S. Chase , Darrell C. Anderson , Prachi N. Thakar , Amin M. Vahdat , Ronald P. Doyle, Managing energy and server resources in hosting centers, Proceedings of the eighteenth ACM symposium on Operating systems principles, October 21-24, 2001, Banff, Alberta, Canada
• 10: J. Choi, S. Govindan, B. Urgaonkar, and A. Sivasubramaniam. Profiling, prediction, and capping of power consumption in consolidated environments. In Ethan L. Miller and Carey L.Williamson, editors, MASCOTS, pages 3--12. IEEE Computer Society, 2008.
• 11: Christopher Clark , Keir Fraser , Steven Hand , Jacob Gorm Hansen , Eric Jul , Christian Limpach , Ian Pratt , Andrew Warfield, Live migration of virtual machines, Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation, p.273-286, May 02-04, 2005
• 12: Y. Diao, N. Gandhi, J.L. Hellerstein, S. Parekh, and D.M. Tilbury. MIMO control of an Apache Web server: Modeling and controller design. In Proceedings of American Control Conference (ACC), 2002.
• 13: A. Gulati, A. Merchant, M. Uysal, and P.J. Varman. Efficient and adaptive proportional share I/O scheduling. Technical Report HPL-2007-186, HP Labs, November 2007.
• 14: Diwaker Gupta , Ludmila Cherkasova , Rob Gardner , Amin Vahdat, Enforcing performance isolation across virtual machines in Xen, Proceedings of the ACM/IFIP/USENIX 2006 International Conference on Middleware, November 01-01, 2006, Melbourne, Australia
• 15: J. L. Hellerstein. Designing in control engineering of computing systems. In Proceedings of American Control Conference (ACC), 2004.
• 16: J. Heo, X. Zhu, P. Padala, and Z.Wang. Memory overbooking and dynamic control of Xen virtual machines in consolidated environments. In Proceedings of IFIP/IEEE Symposium on Integrated Management (IM'09) mini-conference, June 2009.
• 17: Wei Jin , Jeffrey S. Chase , Jasleen Kaur, Interposed proportional sharing for a storage service utility, Proceedings of the joint international conference on Measurement and modeling of computer systems, June 10-14, 2004, New York, NY, USA
• 18: Michael B. Jones , Daniela Roşu , Marcel-Cătălin Roşu, CPU reservations and time constraints: efficient, predictable scheduling of independent activities, Proceedings of the sixteenth ACM symposium on Operating systems principles, p.198-211, October 05-08, 1997, Saint Malo, France
• 19: A. Kamra, V. Misra, and E. Nahum. Yaksha: A selftuning controller for managing the performance of 3-tiered Web sites. In Proceedings of International Workshop on Quality of Service (IWQoS), June 2004.
• 20: Christos Karamanolis , Magnus Karlsson , Xiaoyun Zhu, Designing controllable computer systems, Proceedings of the 10th conference on Hot Topics in Operating Systems, p.9-9, June 12-15, 2005, Santa Fe, NM
• 21: M. Karlsson, C. Karamanolis, and X. Zhu. Triage: Performance isolation and differentiation for storage systems. In Proceedings of the 12th IEEE International Workshop on Quality of Service (IWQoS), 2004.
• 22: X. Liu, X. Zhu, P. Padala, Z.Wang, and S. Singhal. Optimal multivariate control for differentiated services on a shared hosting platform. In Proceedings of IEEE Conference on Decision and Control (CDC), 2007.
• 23: Ying Lu , Tarek F. Abdelzaher , Avneesh Saxena, Design, Implementation, and Evaluation of Differentiated Caching Services, IEEE Transactions on Parallel and Distributed Systems, v.15 n.5, p.440-452, May 2004 [doi>10.1109/TPDS.2004.1278101]
• 24: Christopher R. Lumb , Arif Merchant , Guillermo A. Alvarez, Façade: Virtual Storage Devices with Performance Guarantees, Proceedings of the 2nd USENIX Conference on File and Storage Technologies, March 31-31, 2003, San Francisco, CA
• 25: Jason Nieh , Monica S. Lam, The design, implementation and evaluation of SMART: a scheduler for multimedia applications, Proceedings of the sixteenth ACM symposium on Operating systems principles, p.184-197, October 05-08, 1997, Saint Malo, France
• 26: P. Padala, K. Hou, X. Zhu, M. Uysal, Z. Wang, S. Singhal, A. Merchant, and K. G. Shin. Automated control of multiple virtualized resources. Technical Report HPL-2008-123, HP Labs, Oct 2008.
• 27: Pradeep Padala , Kang G. Shin , Xiaoyun Zhu , Mustafa Uysal , Zhikui Wang , Sharad Singhal , Arif Merchant , Kenneth Salem, Adaptive control of virtualized resources in utility computing environments, Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007, March 21-23, 2007, Lisbon, Portugal
• 28: Jerry Rolia , Ludmila Cherkasova , Martin Arlitt , Artur Andrzejak, A capacity management service for resource pools, Proceedings of the 5th international workshop on Software and performance, p.229-237, July 12-14, 2005, Palma, Illes Balears, Spain [doi>10.1145/1071021.1071047]
• 29: M. Rosenblum. VMware's Virtual Platform: A virtual machine monitor for commodity PCs. In Hot Chips 11, 1999.
• 30: Kai Shen , Hong Tang , Tao Yang , Lingkun Chu, Integrated resource management for cluster-based Internet services, ACM SIGOPS Operating Systems Review, v.36 n.SI, Winter 2002 [doi>10.1145/844128.844150]
• 31: David C. Steere , Ashvin Goel , Joshua Gruenberg , Dylan McNamee , Calton Pu , Jonathan Walpole, A feedback-driven proportion allocator for real-rate scheduling, Proceedings of the third symposium on Operating systems design and implementation, p.145-158, February 1999, New Orleans, Louisiana, United States
• 32: W. Tang, Y. Fu, L. Cherkasova, and A. Vahdat. Longterm streaming media server workload analysis and modeling. Technical Report HPL-2003-23, HP Labs, February 07 2003.
• 33: Bhuvan Urgaonkar , Prashant Shenoy , Timothy Roscoe, Resource overbooking and application profiling in shared hosting platforms, Proceedings of the 5th symposium on Operating systems design and implementation

  Due to copyright restrictions we are not able to make the PDFs for this conference available for downloading

  , December 09-11, 2002, Boston, Massachusetts [doi>10.1145/1060289.1060312]
• 34: Carl A. Waldspurger , William E. Weihl, Lottery scheduling: flexible proportional-share resource management, Proceedings of the 1st USENIX conference on Operating Systems Design and Implementation, p.1-es, November 14-17, 1994, Monterey, California
• 35: Carl A. Waldspurger, Memory resource management in VMware ESX server, Proceedings of the 5th symposium on Operating systems design and implementation

  Due to copyright restrictions we are not able to make the PDFs for this conference available for downloading

  , December 09-11, 2002, Boston, Massachusetts [doi>10.1145/1060289.1060307]
• 36: Brian White , Jay Lepreau , Leigh Stoller , Robert Ricci , Shashi Guruprasad , Mac Newbold , Mike Hibler , Chad Barb , Abhijeet Joglekar, An integrated experimental environment for distributed systems and networks, Proceedings of the 5th symposium on Operating systems design and implementation

  Due to copyright restrictions we are not able to make the PDFs for this conference available for downloading

  , December 09-11, 2002, Boston, Massachusetts [doi>10.1145/1060289.1060313]
• 37: Timothy Wood , Ludmila Cherkasova , Kivanc Ozonat , Prashant Shenoy, Profiling and modeling resource usage of virtualized applications, Proceedings of the 9th ACM/IFIP/USENIX International Conference on Middleware, December 01-05, 2008, Leuven, Belgium
• 38: T. Wood, P. J. Shenoy, A. Venkataramani, and M. S. Yousif. Black-box and gray-box strategies for virtual machine migration. In Proceedings of the 4th Symposium on Networked Systems Design and Implementation (NSDI). USENIX, 2007.
• 39: Yuting Zhang , Azer Bestavros , Mina Guirguis , Ibrahim Matta , Richard West, Friendly virtual machines: leveraging a feedback-control model for application adaptation, Proceedings of the 1st ACM/USENIX international conference on Virtual execution environments, June 11-12, 2005, Chicago, IL, USA [doi>10.1145/1064979.1064983]

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.8 Performance

Additional Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Modeling techniques

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.8 Problem Solving, Control Methods, and Search
          Subjects: Control theory

General Terms:
Design, Experimentation, Management, Performance

Keywords:
application qos, automated control, control theory, data center, resource management, server consolidation, virtualization