TwinDrivers: semi-automatic derivation of fast and safe hypervisor network drivers from guest OS drivers

ABSTRACT

In a virtualized environment, device drivers are often run inside a virtual machine (VM) rather than in the hypervisor, for reasons of safety and reduction in software engineering effort. Unfortunately, this approach results in poor performance for I/O-intensive devices such as network cards. The alternative approach of running device drivers directly in the hypervisor yields better performance, but results in the loss of safety guarantees for the hypervisor and incurs additional software engineering costs.

In this paper we present TwinDrivers, a framework which allows us to semi-automatically create safe and efficient hypervisor drivers from guest OS drivers. The hypervisor driver runs directly in the hypervisor, but its data resides completely in the driver VM address space. A Software Virtual Memory mechanism allows the driver to access its VM data efficiently from the hypervisor running in any guest context, and also protects the hypervisor from invalid memory accesses from the driver. An upcall mechanism allows the hypervisor to largely reuse the driver support infrastructure present in the VM. The TwinDriver system thus combines most of the performance benefits of hypervisor-based driver approaches with the safety and software engineering benefits of VM-based driver approaches.

Using the TwinDrivers hypervisor driver, we are able to improve the guest domain networking throughput in Xen by a factor of 2.4 for transmit workloads, and 2.1 for receive workloads, both in CPU-scaled units, and achieve close to 64-67 of native Linux throughput.

REFERENCES

• 1: The netperf benchmark. http://www.netperf.org/netperf/NetperfPage.html.
• 2: Specweb'99 benchmark. http://spec.org/web99.
• 3: Paul Barham , Boris Dragovic , Keir Fraser , Steven Hand , Tim Harris , Alex Ho , Rolf Neugebauer , Ian Pratt , Andrew Warfield, Xen and the art of virtualization, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 4: B. N. Bershad , S. Savage , P. Pardyak , E. G. Sirer , M. E. Fiuczynski , D. Becker , C. Chambers , S. Eggers, Extensibility safety and performance in the SPIN operating system, Proceedings of the fifteenth ACM symposium on Operating systems principles, p.267-283, December 03-06, 1995, Copper Mountain, Colorado, United States
• 5: Úlfar Erlingsson , Silicon Valley , Martín Abadi , Michael Vrable , Mihai Budiu , George C. Necula, XFI: software guards for system address spaces, Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation, p.6-6, November 06-08, 2006, Seattle, WA
• 6: Bryan Ford , Godmar Back , Greg Benson , Jay Lepreau , Albert Lin , Olin Shivers, The Flux OSKit: a substrate for kernel and language research, Proceedings of the sixteenth ACM symposium on Operating systems principles, p.38-51, October 05-08, 1997, Saint Malo, France
• 7: K. Fraser, S. Hand, R. Neugebauer, I. Pratt, A. Warfield, and M. Williamson. Safe hardware access with the Xen virtual machine monitor. In 1st Workshop on Operating System and Architectural Support for the on demand IT InfraStructure (OASIS), Oct 2004.
• 8: Vinod Ganapathy , Matthew J. Renzelmann , Arini Balakrishnan , Michael M. Swift , Somesh Jha, The design and implementation of microdrivers, Proceedings of the 13th international conference on Architectural support for programming languages and operating systems, March 01-05, 2008, Seattle, WA, USA
• 9: Jorrit N. Herder , Herbert Bos , Ben Gras , Philip Homburg , Andrew S. Tanenbaum, MINIX 3: a highly reliable, self-repairing operating system, ACM SIGOPS Operating Systems Review, v.40 n.3, July 2006 [doi>10.1145/1151374.1151391]
• 10: Joshua LeVasseur , Volkmar Uhlig , Jan Stoess , Stefan Götz, Unmodified device driver reuse and improved system dependability via virtual machines, Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation, p.2-2, December 06-08, 2004, San Francisco, CA
• 11: Aravind Menon , Alan L. Cox , Willy Zwaenepoel, Optimizing network virtualization in Xen, Proceedings of the annual conference on USENIX '06 Annual Technical Conference, p.2-2, May 30-June 03, 2006, Boston, MA
• 12: Aravind Menon , Jose Renato Santos , Yoshio Turner , G. (John) Janakiraman , Willy Zwaenepoel, Diagnosing performance overheads in the xen virtual machine environment, Proceedings of the 1st ACM/USENIX international conference on Virtual execution environments, June 11-12, 2005, Chicago, IL, USA [doi>10.1145/1064979.1064984]
• 13: Aravind Menon , Willy Zwaenepoel, Optimizing TCP receive performance, USENIX 2008 Annual Technical Conference on Annual Technical Conference, p.85-98, June 22-27, 2008, Boston, Massachusetts
• 14: D. Mosberger and T. Jin. httperf: A tool for measuring web server performance. In First Workshop on Internet Server Performance, pages 59--67, Madison, WI, June 1998.
• 15: Jose Renato Santos , Yoshio Turner , G. Janakiraman , Ian Pratt, Bridging the gap between software and hardware techniques for I/O virtualization, USENIX 2008 Annual Technical Conference on Annual Technical Conference, p.29-42, June 22-27, 2008, Boston, Massachusetts
• 16: Margo I. Seltzer , Yasuhiro Endo , Christopher Small , Keith A. Smith, Dealing with disaster: surviving misbehaved kernel extensions, Proceedings of the second USENIX symposium on Operating systems design and implementation, p.213-227, October 29-November 01, 1996, Seattle, Washington, United States
• 17: Michael M. Swift , Brian N. Bershad , Henry M. Levy, Improving the reliability of commodity operating systems, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 18: Rob von Behren , Jeremy Condit , Feng Zhou , George C. Necula , Eric Brewer, Capriccio: scalable threads for internet services, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 19: Robert Wahbe , Steven Lucco , Thomas E. Anderson , Susan L. Graham, Efficient software-based fault isolation, Proceedings of the fourteenth ACM symposium on Operating systems principles, p.203-216, December 05-08, 1993, Asheville, North Carolina, United States
• 20: Carl A. Waldspurger, Memory resource management in VMware ESX server, Proceedings of the 5th symposium on Operating systems design and implementation

  Due to copyright restrictions we are not able to make the PDFs for this conference available for downloading

  , December 09-11, 2002, Boston, Massachusetts [doi>10.1145/1060289.1060307]
• 21: Paul Willmann , Jeffrey Shafer , David Carr , Aravind Menon , Scott Rixner , Alan L. Cox , Willy Zwaenepoel, Concurrent Direct Network Access for Virtual Machine Monitors, Proceedings of the 2007 IEEE 13th International Symposium on High Performance Computer Architecture, p.306-317, February 10-14, 2007 [doi>10.1109/HPCA.2007.346208]

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.8 Performance
          Subjects: Measurements

General Terms:
Measurement, Performance

Keywords:
virtualization