An analysis of internet content delivery systems

ABSTRACT

In the span of only a few years, the Internet has experienced an astronomical increase in the use of specialized content delivery systems, such as content delivery networks and peer-to-peer file sharing systems. Therefore, an understanding of content delivery on the Internet now requires a detailed understanding of how these systems are used in practice.This paper examines content delivery from the point of view of four content delivery systems: HTTP web traffic, the Akamai content delivery network, and Kazaa and Gnutella peer-to-peer file sharing traffic. We collected a trace of all incoming and outgoing network traffic at the University of Washington, a large university with over 60,000 students, faculty, and staff. From this trace, we isolated and characterized traffic belonging to each of these four delivery classes. Our results (1) quantify the rapidly increasing importance of new content delivery systems, particularly peer-to-peer networks, (2) characterize the behavior of these systems from the perspectives of clients, objects, and servers, and (3) derive implications for caching in these systems.

REFERENCES

• 1: Akamai. http://www. akamai. com.
• 2: Jussara Almeida , Virgilio Almeida , David Yates, Measuring the Behavior of a World-Wide Web Server, Boston University, Boston, MA, 1996
• 3: L. Breslau, P. Cao, L. Fan, G. Phillips, and S. Shenker. Web caching and zipf-like distributions: Evidence and implications. In Proc. of IEEE INFOCOM 1999, March 1999.
• 4: R. Caceres, F. Douglis, A. Feldmann, G. Glass, and M. Rabinovich. Web proxy caching: The devil is in the details. In Workshop on Internet Server Performance, June 1998.
• 5: P. Cao, J. Zhang, and K. Beach. Active cache: Caching dynamic contents on the web. In Proc. of IFIP Int. Conf. on Distributed Systems Platforms and Open Distributed Processing, Sep. 1998.
• 6: Soumen Chakrabarti , Byron E. Dom , S. Ravi Kumar , Prabhakar Raghavan , Sridhar Rajagopalan , Andrew Tomkins , David Gibson , Jon Kleinberg, Mining the Web's Link Structure, Computer, v.32 n.8, p.60-67, August 1999 [doi>10.1109/2.781636]
• 7: A. Chankhunthod, P. B. Danzig, C. Neerdaels, M. F. Schwartz, and K. J. Worrell. A hierarchical internet object cache. In Proc. of the 1996 USENIX Annual Technical Conf., Jan. 1996.
• 8: M. Chesire, A. Wolman, G. Voelker, and H. Levy. Measurement and analysis of a streaming media workload. In Proc. of the 2001 USENIX Symp. on Internet Technologies and Systems, March 2001.
• 9: Ian Clarke , Oskar Sandberg , Brandon Wiley , Theodore W. Hong, Freenet: a distributed anonymous information storage and retrieval system, International workshop on Designing privacy enhancing technologies: design issues in anonymity and unobservability, p.46-66, January 2001, Berkeley, California, United States
• 10: Clip2. The Gnutella protocol specification v.0.4, March 2001. http://www.clip2.com/GnutellaProtocol04.pdf.
• 11: F. Douglis, A. Feldmann, B. Krishnamurthy, and J. C. Mogul. Rate of change and other metrics: a live study of the world wide web. In Proc. of the 1997 USENIX Symp. on Internet Technologies and Systems, Dec. 1997.
• 12: B. Duska, D. Marwood, and M. J. Feeley. The measured access characteristics of World Wide Web client proxy caches. In Proc. of the 1st USENIX Symp. on Internet Technologies and Systems, Dec. 1997.
• 13: Li Fan , Pei Cao , Jussara Almeida , Andrei Z. Broder, Summary cache: a scalable wide-area Web cache sharing protocol, Proceedings of the ACM SIGCOMM '98 conference on Applications, technologies, architectures, and protocols for computer communication, p.254-265, August 31-September 04, 1998, Vancouver, British Columbia, Canada
• 14: S. Gadde, J. Chase, and M. Rabinovich. Web caching and content distribution: A view from the interior. In Proc. of the 5th International Web Caching and Content Delivery Workshop, May 2000.
• 15: S. D. Gribble and E. A. Brewer. System design issues for internet middleware services: Deductions from a large client trace. In Proc. of the 1997 USENIX Symp. on Internet Technologies and Systems, Dec. 1997.
• 16: Internet Engineering Task Force. Hypertext transfer protocol - http 1.1. RFC 2068, March 1997.
• 17: Internet2. http://netflow.internet2.edu/weekly/20020422.
• 18: K. L. Johnson, J. F. Carr, M. S. Day, and M. Frans Kaashoek. The measured performance of content distribution networks. Computer Communications, 24(2), 2001.
• 19: J. Kangasharju, K. W. Ross, and J. W. Roberts. Performance evaluation of redirection schemes in content distribution networks. Computer Communications, 24(2):207--214, 2001.
• 20: Kazaa. http://www.kazaa.com.
• 21: J. M. Kleinberg, R. Kumar, P. Raghavan, S. Rajagopalan, and A. S. Tomkins. The Web as a graph: Measurements, models, and methods. In T. Asano, H. Imai, D. T. Lee, S. Nakano, and T. Tokuyama, editors, Proc. of the 5th Annual Int. Conf. Computing and Combinatorics, number 1627. Springer-Verlag, 1999.
• 22: M. Koletsou and G. M. Voelker. The Medusa proxy: A tool for exploring user-perceived web performance. In Proc. of the Sixth Int. Workshop on Web Caching and Content Distribution, June 2001.
• 23: Balachander Krishnamurthy , Craig Wills , Yin Zhang, On the use and performance of content distribution networks, Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement, November 01-02, 2001, San Francisco, California, USA [doi>10.1145/505202.505224]
• 24: N. Leibowitz, A. Bergman, R. Ben-Shaul, and A. Shavit. Are file swapping networks cacheable? Characterizing P2P traffic. In Proc. of the 7th Int. WWW Caching Workshop, August 2002.
• 25: B. Maggs. Global Internet Content Delivery. Talk delivered in the Internet and Distributed Systems Seminar at Stanford University. http://www.stanford.edu/class/cs548/abstracts.shtml#bruce.
• 26: S. McCanne and V. Jacobson. The BSD packet filter: A new architecture for user-level packet capture. In Proc. of the USENIX Technical Conf., Winter 1993.
• 27: Jean-Marc Menaud , Valérie Issarny , Michel Banâtre, A New Protocol for Efficient Cooperative Transversal Web Caching, Proceedings of the 12th International Symposium on Distributed Computing, p.288-302, September 24-26, 1998
• 28: Napster. http://www.napster.com.
• 29: Venkata N. Padmanabhan , Lili Qiu, The content and access dynamics of a busy Web site: findings and implications, Proceedings of the conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, p.111-123, August 28-September 01, 2000, Stockholm, Sweden
• 30: M. Rabinovich, J. Chase, and S. Gadde. Not all hits are created equal: Cooperative proxy caching over a wide area network. In Proc. of the 3rd Int. WWW Caching Workshop, June 1998.
• 31: S. Saroiu, P. K. Gummadi, and S. D. Gribble. A measurement study of peer-to-peer file sharing systems. In Proc. of Multimedia Computing and Networking 2002, Jan. 2002.
• 32: A. Shaikh, R. Tewari, and M. Agrawal. On the effectiveness of DNS-based server selection. In Proc. of IEEE INFOCOM 2001, Anchorage, AK, USA 2001.
• 33: Design Considerations for Distributed Caching on the Internet, Proceedings of the 19th IEEE International Conference on Distributed Computing Systems, p.273, May 31-June 04, 1999
• 34: D. Wessels, K. Claffy, and H.-W. Braun. NLANR prototype web caching system. http://ircache.n1anr.net/.
• 35: A. Wolman, G. Voelker, N. Sharma, N. Cardwell, M. Brown, T. Landray, D. Pinnel, A. Karlin, and H. Levy. Organization based analysis of web-object sharing and caching. In Proc. of the 2nd USENIX Conf. on Internet Technologies and Systems, Oct. 1999.
• 36: Alec Wolman , M. Voelker , Nitin Sharma , Neal Cardwell , Anna Karlin , Henry M. Levy, On the scale and performance of cooperative Web proxy caching, Proceedings of the seventeenth ACM symposium on Operating systems principles, p.16-31, December 12-15, 1999, Charleston, South Carolina, United States
• 37: Beverly Yang , Hector Garcia-Molina, Improving Search in Peer-to-Peer Networks, Proceedings of the 22 nd International Conference on Distributed Computing Systems (ICDCS'02), p.5, July 02-05, 2002