Fast networking with socket-outsourcing in hosted virtual machine environments

ABSTRACT

This paper proposes a novel method of achieving fast networking in hosted virtual machine (VM) environments. This method, called socket-outsourcing, replaces the socket layer in a guest operating system (OS) with the socket layer of the host OS. Socket-outsourcing increases network performance by eliminating duplicate message copying in both the host OS and the guest OS. Furthermore, socket-outsourcing significantly enhances inter-VM communication within the same host OS since it enables network packets to bypass the protocol stack in guest OSes. Socket-outsourcing was implemented in two representative operating systems (Linux and NetBSD) and on two virtual machine monitors (Linux KVM and PansyVM). These virtual machine monitors provided support for socket-outsourcing through shard memory, event queues, and VM-specific Remote Procedure Call between a guest OS and a host OS. The experimental results revealed that a guest OS outsourcing the socket layer achieved the same network throughput as a native OS using up to four Gigabit Ethernet links. Moreover, the benchmark results obtained from an N-tier Web application that generated a significant amount of inter-VM communication indicated that socket-outsourcing improved performance by up to 45 percent compared with conventional hosted VM environments.

REFERENCES

• 1: Dan Aloni: "Cooperative Linux", the Ottawa Linux Symposium (OLS-2004), pp. 23--31 (2004).
• 2: Fabrice Bellard, QEMU, a fast and portable dynamic translator, Proceedings of the annual conference on USENIX Annual Technical Conference, p.41-41, April 10-15, 2005, Anaheim, CA
• 3: Emmanuel Cecchet, Julie Marguerite, and Willy Zwaenepoel: "Performance and scalability of EJB applications", Object Oriented Programming Systems Languages and Applications (OOPSLA), pp. 246--261 (2002).
• 4: Jeff Dike, A user-mode port of the linux kernel, Proceedings of the 4th annual Linux Showcase & Conference, p.7-7, October 10-14, 2000, Atlanta, Georgia
• 5: Hideki Eiraku , Yasushi Shinjo, Running BSD kernels as user processes by partial emulation and rewriting of machine instructions, Proceedings of the BSD Conference 2003 on BSD Conference, p.10-10, September 08-12, 2003, San Mateo, California
• 6: Doug Freimuth , Elbert Hu , Jason LaVoie , Ronald Mraz , Erich Nahum , Prashant Pradhan , John Tracey, Server network scalability and TCP offload, Proceedings of the annual conference on USENIX Annual Technical Conference, p.15-15, April 10-15, 2005, Anaheim, CA
• 7: Mark Gates, Ajay Tirumala, Jon Dugan, and Kevin Gibbs: "Iperf User Docs" (2003). http://sourceforge.net/projects/iperf
• 8: Robert P. Goldberg: "Survey of Virtual Machine Research. IEEE Computer, pp. 34--45 (1974).
• 9: Paul Barham , Boris Dragovic , Keir Fraser , Steven Hand , Tim Harris , Alex Ho , Rolf Neugebauer , Ian Pratt , Andrew Warfield, Xen and the art of virtualization, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 10: Kangho Kim , Cheiyol Kim , Sung-In Jung , Hyun-Sup Shin , Jin-Soo Kim, Inter-domain socket communications supporting high performance and full binary compatibility on Xen, Proceedings of the fourth ACM SIGPLAN/SIGOPS international conference on Virtual execution environments, March 05-07, 2008, Seattle, WA, USA [doi>10.1145/1346256.1346259]
• 11: Krishna Kant: "TCP Offload Performance for Front-End Servers", IEEE Global Telecommunications Conference (GLOBECOM 03), pp. 3242--324 (2003).
• 12: Samuel T. King , George W. Dunlap , Peter M. Chen, Operating system support for virtual machines, Proceedings of the annual conference on USENIX Annual Technical Conference, p.6-6, June 09-14, 2003, San Antonio, Texas
• 13: Avi Kivity, Yaniv Kamay, and Dor Laor: "kvm: the Linux Virtual Machine Monitor", the Linux Symposium, pp. 225--230 (2007).
• 14: Younggyun Koh, Calton Pu, Sapan Bhatia, and Charles Consel: "Efficient Packet Processing in User-Level OSes: A Study of UML", the 31st IEEE Conference on Local Computer Networks (2006).
• 15: Marshall Kirk McKusick , Keith Bostic , Michael J. Karels , John S. Quarterman, The design and implementation of the 4.4BSD operating system, Addison Wesley Longman Publishing Co., Inc., Redwood City, CA, 1996
• 16: Aravind Menon , Alan L. Cox , Willy Zwaenepoel, Optimizing network virtualization in Xen, Proceedings of the annual conference on USENIX '06 Annual Technical Conference, p.2-2, May 30-June 03, 2006, Boston, MA
• 17: John Scott Robin , Cynthia E. Irvine, Analysis of the Intel Pentium's ability to support a secure virtual machine monitor, Proceedings of the 9th conference on USENIX Security Symposium, p.10-10, August 14-17, 2000, Denver, Colorado
• 18: Murali Rangarajan, Aniruddha Bohra, Kalpana Banerjee, Enrique V. Carrera, Ricardo Bianchini, Liviu Iftode, and Willy Zwaenepoel: "TCP Servers: Offloading TCP/IP Processing in Internet Servers. Design, Implementation, and Performance", Computer Science Department, Rutgers University, Technical Report DCR-TR-48 (2002).
• 19: Greg Regnier , Srihari Makineni , Ramesh Illikkal , Ravi Iyer , Dave Minturn , Ram Huggahalli , Don Newell , Linda Cline , Annie Foong, TCP Onloading for Data Center Servers, Computer, v.37 n.11, p.48-58, November 2004 [doi>10.1109/MC.2004.223]
• 20: Mendel Rosenblum , Tal Garfinkel, Virtual Machine Monitors: Current Technology and Future Trends, Computer, v.38 n.5, p.39-47, May 2005 [doi>10.1109/MC.2005.176]
• 21: Rusty Russell, virtio: towards a de-facto standard for virtual I/O devices, ACM SIGOPS Operating Systems Review, v.42 n.5, p.95-103, July 2008 [doi>10.1145/1400097.1400108]
• 22: Sunghoon Son, Jaeyol Kim, Eunji Lim, and Sungin Jung: "SOP: A Socket Interface for TOEs", Internet and Multimedia Systems and Applications, pp. 294--299 (2004).
• 23: Jeremy Sugerman , Ganesh Venkitachalam , Beng-Hong Lim, Virtualizing I/O Devices on VMware Workstation's Hosted Virtual Machine Monitor, Proceedings of the General Track: 2002 USENIX Annual Technical Conference, p.1-14, June 25-30, 2001
• 24: "Paravirtualization API Version 2.0", VMware, http://www.vmware.com (2006).
• 25: Carl A. Waldspurger, Memory resource management in VMware ESX server, Proceedings of the 5th symposium on Operating systems design and implementation

  Due to copyright restrictions we are not able to make the PDFs for this conference available for downloading

  , December 09-11, 2002, Boston, Massachusetts [doi>10.1145/1060289.1060307]
• 26: Jian Wang , Kwame-Lante Wright , Kartik Gopalan, XenLoop: a transparent high performance inter-vm network loopback, Proceedings of the 17th international symposium on High performance distributed computing, June 23-27, 2008, Boston, MA, USA [doi>10.1145/1383422.1383437]
• 27: Xiaolan Zhang , Suzanne McIntosh , Pankaj Rohatgi , John Linwood Griffin, XenSocket: a high-throughput interdomain transport for virtual machines, Proceedings of the ACM/IFIP/USENIX 2007 International Conference on Middleware, November 26-30, 2007, Newport Beach, California

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.4 Communications Management

General Terms:
Design, Performance

Keywords:
guest operating systems, host operating systems, hosted virtual machine monitors, outsourcing, paravirtualization, socket API, virtualization