Architecture and design of high volume web sites: (a brief history of IBM sport and event web sites)

ABSTRACT

Architecting and designing high volume Web sites has changed immensely over the last six years. These changes include the availability of inexpensive Pentium based servers, Linux, Java applications, commodity switches, connection management and caching engines, bandwidth price reductions, content distribution services, and many others. This paper describes the evolution of the best practices within IBM in architecting sites that handle millions of page views per day. Discussed is the transition to multi-tiered architectures, the use of publish/subscribe software to reduce Web site hits, the migration from static to dynamic content, and techniques for caching of dynamic and personalized content.

REFERENCES

• 1: An Efficient Multicast Protocol for Content-Based Publish-Subscribe Systems, Proceedings of the 19th IEEE International Conference on Distributed Computing Systems, p.262, May 31-June 04, 1999
• 2: Jim Challenger , Paul Dantzig , Arun Iyengar, A scalable and highly available system for serving dynamic data at frequently accessed web sites, Proceedings of the 1998 ACM/IEEE conference on Supercomputing (CDROM), p.1-30, November 07-13, 1998, San Jose, CA
• 3: Challenger, J. Dantzig, P. M., Iyengar, A. A Scalable System for Consistently Caching Dynamic Web Data, Proceedings of IEEE INFOCOM'99, 1999
• 4: Challenger, J. Ferstat, C. Iyengar, A., Reed, P. Witting, C. A Publishing System for Efficiently Creating Dynamic Web Content, Proceedings of IEEE INFOCOM'00,2000
• 5: Challenger, J. Iyengar, A. Jin, S. Techniques for Allocating Persistent Storage, IEEE Symposium on Large Scale Storage in the Web, April 2001.
• 6: M. Colajanni , P. S. Yu , D. M. Dias, Scheduling algorithms for distributed Web servers, Proceedings of the 17th International Conference on Distributed Computing Systems (ICDCS '97), p.169, May 27-30, 1997
• 7: Yew-Huey Liu , P. Dantzig , C. E. Wu , L. M. Ni, A distributed connection manager interface for web services on IBM SP systems, Proceedings of the 1996 International Conference on Parallel and Distributed Systems, p.2, June 03-06, 1996
• 8: D. M. Dias , W. Kish , R. Mukherjee , R. Tewari, A scalable and highly available web server, Proceedings of the 41st IEEE International Computer Conference, p.85, February 25-28, 1996
• 9: Guinness World Records, Guinness Book of Records, 1998.
• 10: Java (TM Sun Microsystems) Message Service Documentation, http://java.sun.com/products/jms/docs.html
• 11: 1996 Atlanta Olympic Games Web Site, July 19 - August 04, 1996, http://www.atlanta.olympic.org, site is not available.
• 12: 1998 Nagano Olympic Games Web Site, February 7 - February 22, 1998,http://www.nagano.olympic.org, site is not available.
• 13: 1999 The Championships Wimbledon, June 21 - July 4, 1999, http://www.wimbledon.org, site is not available.
• 14: 2000 Sydney Olympic Games Web Site, September 15 - October 1, 2000, http://www.olympic.com, site is not available.
• 15: 2001 The Championships Wimbledon, June 25 - July 8, 2001, http://www.wimbledon.org, site is not available.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.5 On-line Information Services

General Terms:
Design, Performance, Reliability

Keywords:
content distribution, content management, java messaging services, proxy caching, publish and subscribe, web serving