Building a robust software-based router using network processors

ABSTRACT

Recent efforts to add new services to the Internet have increased interest in software-based routers that are easy to extend and evolve. This paper describes our experiences using emerging network processors---in particular, the Intel IXP1200---to implement a router. We show it is possible to combine an IXP1200 development board and a PC to build an inexpensive router that forwards minimum-sized packets at a rate of 3.47Mpps. This is nearly an order of magnitude faster than existing pure PC-based routers, and sufficient to support 1.77Gbps of aggregate link bandwidth. At lesser aggregate line speeds, our design also allows the excess resources available on the IXP1200 to be used robustly for extra packet processing. For example, with 8 × 100Mbps links, 240 register operations and 96 bytes of state storage are available for each 64-byte packet. Using a hierarchical architecture we can guarantee line-speed forwarding rates for simple packets with the IXP1200, and still have extra capacity to process exceptional packets with the Pentium. Up to 310Kpps of the traffic can be routed through the Pentium to receive 1510 cycles of extra per-packet processing.

REFERENCES

• 1: Matthias A. Blumrich , Cezary Dubnicki , Edward W. Felten , Kai Li , Malena R. Mesarina, Virtual-Memory-Mapped Network Interfaces, IEEE Micro, v.15 n.1, p.21-28, February 1995 [doi>10.1109/40.342014]
• 2: A. T. Campbell, S. Chou, M. E. Konnavis, and V. D. Stachtos. Implementing Routelets: Virtual Router Support for the IXP1200 Network Processor. In/XA Univeristy Program Workshop, Portland, Oregon, June 2001.
• 3: M. Dasen, G. Fankhauser, and B. Plattner. An Error Tolerant, Scalable Video Stream Encoding and Compression for Mobile Computing. In Proceedings of ACTS Mobile Summit 96, pages 762-771, November 1996.
• 4: Bruce Davie , Yakov Rekhter, MPLS: technology and applications, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2000
• 5: Dan Decasper , Zubin Dittia , Guru Parulkar , Bernhard Plattner, Router plugins: a software architecture for next-generation routers, IEEE/ACM Transactions on Networking (TON), v.8 n.1, p.2-15, Feb. 2000 [doi>10.1109/90.836474]
• 6: Peter Druschel , Larry L. Peterson , Bruce S. Davie, Experiences with a high-speed network adaptor: a software perspective, Proceedings of the conference on Communications architectures, protocols and applications, p.2-13, August 31-September 02, 1994, London, United Kingdom
• 7: M. E. Fiuczynski, R. E Martin, T. Owa, and B. N. Bershad. On Using Intelligent Network Interface Cards to support Multimedia Applications. In Proceedings of the 8th International Workshop on Network and Operating System Support for Digital Audio and Video, pages 95-98, July 1998.
• 8: IBM Microelectronics Division. IBM PowerNP NP4GS3 Network Processor Solutions Product Overview, April 2001.
• 9: IEEE. Standard 802.3, October 2000.
• 10: Intel Corporation. IXP12OONetworkProcessorDatasheet, September 2000.
• 11: Intelligent I/O (I20) Special Interest Group. Intelligent I/O (I20) Architecture Specification, Version 2.0, March 1999.
• 12: S. Karlin and L. Peterson. VERA: An Extensible Router Architecture. In Proceedings of the 4th International Conference on Open Architectures and Network Programming (OPENARCH), pages 3-14, April 2001.
• 13: Eddie Kohler , Robert Morris , Benjie Chen , John Jannotti , M. Frans Kaashoek, The click modular router, ACM Transactions on Computer Systems (TOCS), v.18 n.3, p.263-297, Aug. 2000 [doi>10.1145/354871.354874]
• 14: M. E. Kounavis, A. T. Campell, S. Chou, E Modoux, J. Vicente, and H. Zhuang. The Genesis Kernel: A Programming System for Spawning Network Architectures. IEEE Journal on Selected Areas in Communications, 19(3):511-526, March 2001.
• 15: Hugh C. Lauer , Roger M. Needham, On the duality of operating system structures, ACM SIGOPS Operating Systems Review, v.13 n.2, p.3-19, April 1979 [doi>10.1145/850657.850658]
• 16: D. L. Mills, The fuzzball, Symposium proceedings on Communications architectures and protocols, p.115-122, August 16-18, 1988, Stanford, California, United States
• 17: Vern Paxson, Automated packet trace analysis of TCP implementations, Proceedings of the ACM SIGCOMM '97 conference on Applications, technologies, architectures, and protocols for computer communication, p.167-179, September 14-18, 1997, Cannes, France
• 18: Prashant Pradhan , Tzi-Cker Chiueh, Operating Systems Support for Programmable Cluster-Based Internet Routers, Proceedings of the The Seventh Workshop on Hot Topics in Operating Systems, p.76, March 28-30, 1999
• 19: Xiaohu Qie , Andy Bavier , Larry Peterson , Scott Karlin, Scheduling computations on a software-based router, Proceedings of the 2001 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.13-24, June 2001, Cambridge, Massachusetts, United States
• 20: Marcus J. Ranum , Kent Landfield , Michael T. Stolarchuk , Mark Sienkiewicz , Andrew Lambeth , Eric Wall, Implementing a Generalized Tool for Network Monitoring, Proceedings of the 11th Conference on Systems Administration, p.1-8, October 26-31, 1997
• 21: Oliver Spatscheck , Jørgen S. Hansen , John H. Hartman , Larry L. Peterson, Optimizing TCP forwarder performance, IEEE/ACM Transactions on Networking (TON), v.8 n.2, p.146-157, April 2000 [doi>10.1109/90.842138]
• 22: V. Srinivasan , G. Varghese, Fast address lookups using controlled prefix expansion, ACM Transactions on Computer Systems (TOCS), v.17 n.1, p.1-40, Feb. 1999 [doi>10.1145/296502.296503]
• 23: D. E. Taylor, J. S. Turner, and J. W. Lockwood. Dynamic Hardware Plugins (DHP): Exploiting Reconfigurable Hardware for High-Performance Programmable Routers. In Proceedings of the 4th International Conference on Open Architectures and Network Programming ( OPENARCH), pages 25-34, April 2001.
• 24: C. B. S. Traw and J. M. Smith. Hardware/Software Organization of a High-Performance ATM Host Interface. 1EEE Journal on Selected Areas in Communications (Special Issue on High Speed Computer/Network Interfaces), 11(2):240-253, 1993.
• 25: Vitesse Semiconductor Corporation. IQ2000 Network Processor Product Brief, 2000.
• 26: T. von Eicken , A. Basu , V. Buch , W. Vogels, U-Net: a user-level network interface for parallel and distributed computing (includes URL), Proceedings of the fifteenth ACM symposium on Operating systems principles, p.40-53, December 03-06, 1995, Copper Mountain, Colorado, United States

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.7 Organization and Design
          Subjects: Distributed systems

Additional Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.4 Communications Management
          Subjects: Network communication

General Terms:
Design, Performance