Increasing web server throughput with network interface data caching

ABSTRACT

This paper introduces network interface data caching, a new technique to reduce local interconnect traffic on networking servers by caching frequently-requested content on a programmable network interface. The operating system on the host CPU determines which data to store in the cache and for which packets it should use data from the cache. To facilitate data reuse across multiple packets and connections, the cache only stores application-level response content (such as HTTP data), with application-level and networking headers generated by the host CPU. Network interface data caching can reduce PCI traffic by up to 57% on a prototype implementation of a uniprocessor web server. This traffic reduction results in up to 31% performance improvement, leading to a peak server throughput of 1571 Mb/s.

REFERENCES

• 1: Alteon WebSystems. Gigabit Ethernet/PCI Network Interface Card: Host/NIC Software Interface Definition, July 1999. Revision 12.4.13.
• 2: Martin F. Arlitt , Carey L. Williamson, Internet Web servers: workload characterization and performance implications, IEEE/ACM Transactions on Networking (TON), v.5 n.5, p.631-645, Oct. 1997 [doi>10.1109/90.649565]
• 3: Gaurav Banga, Jeffrey C. Mogul, and Peter Druschel. A scalable and explicit event delivery mechanism for UNIX. In Proceedings of the USENIX 1999 Annual Technical Conference, pages 253-265, June 1999.
• 4: Lee Breslau, Pei Cao, Li Fan, Graham Phillips, and Scott Schenker. Web Caching and Zipf-like Distributions: Evidence and Implications. In Proceedings of IEEE INFOCOM '99, volume 1, pages 126-134, March 1999.
• 5: Philip Buonadonna , David Culler, Queue pair IP: a hybrid architecture for system area networks, Proceedings of the 29th annual international symposium on Computer architecture, p.247, May 25-29, 2002, Anchorage, Alaska
• 6: Pei Cao and Sandy Irani. Cost-Aware WWW Proxy Caching Algorithms. In Proceedings of the 1997 USENIX Symposium on Internet Technology and Systems, pages 193-206, December 1997.
• 7: Peter Druschel , Larry L. Peterson, Fbufs: a high-bandwidth cross-domain transfer facility, Proceedings of the fourteenth ACM symposium on Operating systems principles, p.189-202, December 05-08, 1993, Asheville, North Carolina, United States
• 8: R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, and T. Berners-Lee. Hypertext Transfer Protocol - HTTP 1.1. IETF RFC 2616, June 1999.
• 9: Karl Kleinpaste , Peter Steenkiste , Brian Zill, Software support for outboard buffering and checksumming, Proceedings of the conference on Applications, technologies, architectures, and protocols for computer communication, p.87-98, August 28-September 01, 1995, Cambridge, Massachusetts, United States
• 10: Raj Krishnamurthy , Karsten Schwan , Richard West , Marcel-Catalin Rosu, A Network Co-processor-Based Approach to Scalable Media Streaming in Servers, Proceedings of the Proceedings of the 2000 International Conference on Parallel Processing, p.125, August 21-24, 2000
• 11: Erich Nahum , Tsipora Barzilai , Dilip D. Kandlur, Performance issues in WWW servers, IEEE/ACM Transactions on Networking (TON), v.10 n.1, p.2-11, February 2002 [doi>10.1109/90.986497]
• 12: Vivek S. Pai, Peter Druschel, and Willy Zwaenepoel. Flash: An Efficient and Portable Web Server. In Proceedings of the USENIX 1999 Annual Technical Conference, pages 199-212, June 1999.
• 13: Vivek S. Pai , Peter Druschel , Willy Zwaenepoel, IO-lite: a unified I/O buffering and caching system, Proceedings of the third symposium on Operating systems design and implementation, p.15-28, February 1999, New Orleans, Louisiana, United States
• 14: Fabrizio Petrini , Wu-chun Feng , Adolfy Hoisie , Salvador Coll , Eitan Frachtenberg, The Quadrics Network: High-Performance Clustering Technology, IEEE Micro, v.22 n.1, p.46-57, January 2002 [doi>10.1109/40.988689]
• 15: Jef Poskanzer. thttpd - tiny/turbo/throttling HTTP server. Acme Labs, February 2000. Unix manual page.
• 16: Gary Tyson , Matthew Farrens , John Matthews , Andrew R. Pleszkun, A modified approach to data cache management, Proceedings of the 28th annual international symposium on Microarchitecture, p.93-103, November 29-December 01, 1995, Ann Arbor, Michigan, United States
• 17: Darryl L. Willick, Derek L. Eager, and Richard B. Bunt. Disk Cache Replacement Policies for Network Fileservers. In Proceedings of the 13th International Conference on Distributed Computing Systems, pages 2-11, May 1993.
• 18: Ken Yocum , Jeffrey S. Chase, Payload Caching: High-Speed Data Forwarding for Network Intermediaries, Proceedings of the General Track: 2002 USENIX Annual Technical Conference, p.305-317, June 25-30, 2001