Scale and performance in the Denali isolation kernel

ABSTRACT

This paper describes the Denali isolation kernel, an operating system architecture that safely multiplexes a large number of untrusted Internet services on shared hardware. Denali's goal is to allow new Internet services to be "pushed" into third party infrastructure, relieving Internet service authors from the burden of acquiring and maintaining physical infrastructure. Our isolation kernel exposes a virtual machine abstraction, but unlike conventional virtual machine monitors, Denali does not attempt to emulate the underlying physical architecture precisely, and instead modifies the virtual architecture to gain scale, performance, and simplicity of implementation. In this paper, we first discuss design principles of isolation kernels, and then we describe the design and implementation of Denali. Following this, we present a detailed evaluation of Denali, demonstrating that the overhead of virtualization is small, that our architectural choices are warranted, and that we can successfully scale to more than 10,000 virtual machines on commodity hardware.

REFERENCES

• 1: M. Accetta et al. Mach: A new kernel foundation for UNIX development. In Proceedings of the USENIX Summer Conference, 1986.
• 2: Collaborative advanced interagency research network (cairn). http://www.cairn.net, 1997.
• 3: D. Balfanz and D.R. Simon. Windowbox: A simple security model for the connected desktop. In Proceedings of the 4th USENIX Windows Systems Symposium, August 2000.
• 4: Gaurav Banga , Peter Druschel , Jeffrey C. Mogul, Resource containers: a new facility for resource management in server systems, Proceedings of the third symposium on Operating systems design and implementation, p.45-58, February 1999, New Orleans, Louisiana, United States
• 5: L. Breslau et al. Web caching, and Zipf-like distributions: Evidence, and implications, Mar 1999.
• 6: Edouard Bugnion , Scott Devine , Mendel Rosenblum, Disco: running commodity operating systems on scalable multiprocessors, Proceedings of the sixteenth ACM symposium on Operating systems principles, p.143-156, October 05-08, 1997, Saint Malo, France
• 7: Richard W. Carr , John L. Hennessy, WSCLOCK—a simple and effective algorithm for virtual memory management, Proceedings of the eighth ACM symposium on Operating systems principles, p.87-95, December 14-16, 1981, Pacific Grove, California, United States
• 8: Andy Chou , Junfeng Yang , Benjamin Chelf , Seth Hallem , Dawson Engler, An empirical study of operating systems errors, Proceedings of the eighteenth ACM symposium on Operating systems principles, October 21-24, 2001, Banff, Alberta, Canada
• 9: R. J. Creasy. The origin of the VM/370 time-sharing system. IBM Journal of Research and Development, 25(5), 1981.
• 10: Richard P. Draves , Brian N. Bershad , Richard F. Rashid , Randall W. Dean, Using continuations to implement thread management and communication in operating systems, Proceedings of the thirteenth ACM symposium on Operating systems principles, p.122-136, October 13-16, 1991, Pacific Grove, California, United States
• 11: Peter Druschel , Gaurav Banga, Lazy receiver processing (LRP): a network subsystem architecture for server systems, Proceedings of the second USENIX symposium on Operating systems design and implementation, p.261-275, October 29-November 01, 1996, Seattle, Washington, United States
• 12: D. Ely, S. Savage, and D. Wetherall. Alpine: A user-level infrastructure for network protocol development. In Proceedings of the Third USENIX Symposium on Internet Technologies and Systems (USITS '01), March, 2001.
• 13: Bryan Ford , Mike Hibler , Jay Lepreau , Patrick Tullmann , Godmar Back , Stephen Clawson, Microkernels meet recursive virtual machines, Proceedings of the second USENIX symposium on Operating systems design and implementation, p.137-151, October 29-November 01, 1996, Seattle, Washington, United States
• 14: Bryan Ford , Godmar Back , Greg Benson , Jay Lepreau , Albert Lin , Olin Shivers, The Flux OSKit: a substrate for kernel and language research, Proceedings of the sixteenth ACM symposium on Operating systems principles, p.38-51, October 05-08, 1997, Saint Malo, France
• 15: A. F. Forin, D. B. Golub, and B. N. Bershad. An I/O system for Mach. In Proceedings of the Usenix Mach Symposium (MACHNIX), Nov 1991.
• 16: I. Goldberg, D. Wagner, R. Thomas, and E. A. Brewer. A secure environment for untrusted helper applications. In Proceedings of the sixth USENIX Security Symposium, July 1996.
• 17: R. P. Goldberg. Architectural Principles for Virtual Computer Systems. PhD thesis, Harvard University, 1972.
• 18: Dieter Gollmann, Computer security, John Wiley & Sons, Inc., New York, NY, 1999
• 19: Robert S. Gray. Agent Tcl: A Flexible and Secure Mobile-Agent System. In Proceedings of the Fourth Annual Usenix Tcl/Tk Workshop, 1996.
• 20: M. Frans Kaashoek , Dawson R. Engler , Gregory R. Ganger , Hector M. Briceño , Russell Hunt , David Mazières , Thomas Pinckney , Robert Grimm , John Jannotti , Kenneth Mackenzie, Application performance and flexibility on exokernel systems, Proceedings of the sixteenth ACM symposium on Operating systems principles, p.52-65, October 05-08, 1997, Saint Malo, France
• 21: Paul A. Karger , Mary Ellen Zurko , Douglas W. Bonin , Andrew H. Mason , Clifford E. Kahn, A Retrospective on the VAX VMM Security Kernel, IEEE Transactions on Software Engineering, v.17 n.11, p.1147-1165, November 1991 [doi>10.1109/32.106971]
• 22: Jay Lepreau , Bryan Ford , Mike Hibler, The persistent relevance of the local operating system to global applications, Proceedings of the 7th workshop on ACM SIGOPS European workshop: Systems support for worldwide applications, September 09-11, 1996, Connemara, Ireland [doi>10.1145/504450.504477]
• 23: I. Leslie et al. The design and implementation of an operating system to support distributed multimedia applications. IEEE Journal of Selected Areas in Communications, 14(7), 1996.
• 24: R. Meushaw and D. Simard. NetTop: Commercial technology in high assurance applications. http://www.vmware.com/, 2000.
• 25: V. Paxson, J. Mahdavi, A. Adams, and M. Mathis. An architecture for large-scale Internet measurement. IEEE Communications Magazine, 36(8):48--54, August 1998.
• 26: J. Reumann et al. Virtual services: A new abstraction for server consolidation. In Proceedings of the 2000 USENIX Annual Technical Conference, San Diego, USA, June 2000.
• 27: J. S. Robin and C. E. Irvine. Analysis of the Intel Pentium's ability to support a secure virtual machine monitor. In Proceedings of the 9th USENIX Security Symposium, Denver, CO, August 2000.
• 28: J. M. Rushby, Design and verification of secure systems, Proceedings of the eighth ACM symposium on Operating systems principles, p.12-21, December 14-16, 1981, Pacific Grove, California, United States
• 29: J. H. Saltzer and M. D. Schroeder. The protection of information in computer systems. Proceedings of the IEEE, 63(9), 1975.
• 30: Ion Stoica , Robert Morris , David Karger , M. Frans Kaashoek , Hari Balakrishnan, Chord: A scalable peer-to-peer lookup service for internet applications, Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications, p.149-160, August 2001, San Diego, California, United States
• 31: Jeremy Sugerman , Ganesh Venkitachalam , Beng-Hong Lim, Virtualizing I/O Devices on VMware Workstation's Hosted Virtual Machine Monitor, Proceedings of the General Track: 2002 USENIX Annual Technical Conference, p.1-14, June 25-30, 2001
• 32: Michael M. Swift , Peter Brundrett , Cliff Van Dyke , Praerit Garg , Anne Hopkins , Shannon Chan , Mario Goertzel , Gregory Jensenworth, Improving the granularity of access control in Windows NT, Proceedings of the sixth ACM symposium on Access control models and technologies, p.87-96, May 2001, Chantilly, Virginia, United States [doi>10.1145/373256.373271]
• 33: Dan S. Wallach , Dirk Balfanz , Drew Dean , Edward W. Felten, Extensible security architectures for Java, Proceedings of the sixteenth ACM symposium on Operating systems principles, p.116-128, October 05-08, 1997, Saint Malo, France
• 34: A. Wolman et al. Organization-based analysis of web-object sharing and caching. In Proceedings of the 2nd USENIX Conference on Internet Technologies and Systems (USITS '99), Boulder, CO, Oct 1999.

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.9 Systems Programs and Utilities
          Subjects: Window managers

Additional Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.7 Organization and Design
          Subjects: Distributed systems
      D.4.8 Performance
          Subjects: Operational analysis

General Terms:
Design, Management, Performance