Towards high performance virtual routers on commodity hardware

ABSTRACT

Modern commodity hardware architectures, with their multiple multi-core CPUs and high-speed system interconnects, exhibit tremendous power. In this paper, we study performance limitations when building both software routers and software virtual routers on such systems. We show that the fundamental performance bottleneck is currently the memory system, and that through careful mapping of tasks to CPU cores, we can achieve forwarding rates of 7 million minimum-sized packets per second on mid-range server-class systems, thus demonstrating the viability of software routers. We also find that current virtualisation systems, when used to provide forwarding engine virtualisation, yield aggregate performance equivalent to that of a single software router, a tenfold improvement on current virtual router platform performance. Finally, we identify principles for the construction of high-performance software router systems on commodity hardware, including full router virtualisation support.

REFERENCES

• 1: Paul Barham , Boris Dragovic , Keir Fraser , Steven Hand , Tim Harris , Alex Ho , Rolf Neugebauer , Ian Pratt , Andrew Warfield, Xen and the art of virtualization, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 2: "Introducing vmware virtual platform, technical white paper," 1999.
• 3: "Intel vanderpool technology for ia-32 processors (vt-x) preliminary specification," 2005.
• 4: R. J. Creasy, "The origin of the vm/370 time-sharing system." IBM Journal of Research and De-velopment, vol. 25, no. 5, p. 483490, September 1981.
• 5: Andy Bavier , Nick Feamster , Mark Huang , Larry Peterson , Jennifer Rexford, In VINI veritas: realistic and controlled network experimentation, ACM SIGCOMM Computer Communication Review, v.36 n.4, October 2006
• 6: N. Egi, A. Greenhalgh, M. Handley, M. Hoerdt, L. Mathy, and T. Schooley, "Evaluating xen for virtual routers," in PMECT07, August 2007.
• 7: S. Bhatia, M. Motiwala, W. Muhlbauer, V. Valancius, A. Bavier, N. Feamster, L. Peterson, and J. Rexford, "Hosting virtual networks on commodity hardware," Georgia Tech. University., Tech. Rep. GT-CS-07-10, January 2008.
• 8: "Heterogeneous experimental network," http://www.cs.ucl.ac.uk/research/hen/.
• 9: J. Levon and P. E. et al., "Oprofile," http://oprofile.sourceforge.net.
• 10: F. Baker, Requirements for IP Version 4 Routers, RFC Editor, 1995
• 11: OpenVZ Project, "OpenVZ Project," http://www.openvz.org.
• 12: R. Hiremane, "Intel virtualization technology for directed i/o (intel vt-d)," Technology@Intel Magazine, vol. 4, no. 10, May 2007.
• 13: Aravind Menon , Jose Renato Santos , Yoshio Turner , G. (John) Janakiraman , Willy Zwaenepoel, Diagnosing performance overheads in the xen virtual machine environment, Proceedings of the 1st ACM/USENIX international conference on Virtual execution environments, June 11-12, 2005, Chicago, IL, USA [doi>10.1145/1064979.1064984]
• 14: Jose Renato Santos , Yoshio Turner , G. Janakiraman , Ian Pratt, Bridging the gap between software and hardware techniques for I/O virtualization, USENIX 2008 Annual Technical Conference on Annual Technical Conference, p.29-42, June 22-27, 2008, Boston, Massachusetts
• 15: Norbert Egi , Adam Greenhalgh , Mark Handley , Mickael Hoerdt , Felipe Huici , Laurent Mathy, Fairness issues in software virtual routers, Proceedings of the ACM workshop on Programmable routers for extensible services of tomorrow, August 22-22, 2008, Seattle, WA, USA [doi>10.1145/1397718.1397726]
• 16: Katerina Argyraki , Salman Baset , Byung-Gon Chun , Kevin Fall , Gianluca Iannaccone , Allan Knies , Eddie Kohler , Maziar Manesh , Sergiu Nedevschi , Sylvia Ratnasamy, Can software routers scale?, Proceedings of the ACM workshop on Programmable routers for extensible services of tomorrow, August 22-22, 2008, Seattle, WA, USA [doi>10.1145/1397718.1397724]
• 17: Jonathan S. Turner , Patrick Crowley , John DeHart , Amy Freestone , Brandon Heller , Fred Kuhns , Sailesh Kumar , John Lockwood , Jing Lu , Michael Wilson , Charles Wiseman , David Zar, Supercharging planetlab: a high performance, multi-application, overlay network platform, Proceedings of the 2007 conference on Applications, technologies, architectures, and protocols for computer communications, August 27-31, 2007, Kyoto, Japan

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.6 Internetworking
          Subjects: Routers

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.0 General
          Subjects: Data communications
  C.4 PERFORMANCE OF SYSTEMS

General Terms:
Design, Experimentation, Measurement, Performance