A TCP tuning daemon

ABSTRACT

Many high performance distributed applications require high network throughput but are able to achieve only a small fraction of the available bandwidth. A common cause of this problem is improperly tuned network settings. Tuning techniques, such as setting the correct TCP buffers and using parallel streams, are well known in the networking community, but outside the networking community they are infrequently applied. In this paper, we describe a tuning daemon that uses TCP instrumentation data from the Unix kernel to transparently tune TCP parameters for specified individual flows over designated paths. No modifications are required to the application, and the user does not need to understand network or TCP characteristics.

REFERENCES

• 1: W. Allcock, J. Bester, J. Bresnahan, A. Chervenak, I. Foster, C. Kesselman, S. Meder, V. Nefedova, D. Quesnel, and S. Tuecke. "gridFTP", 2000. URL: http://www.globus.org/datagrid/gridftp.html.
• 2: W. Allcock, J. Bester, J. Bresnahan, A. Chervenak, I. Foster, C. Kesselman, S. Meder, V. Nefedova, D. Quesnel, and S. Tuecke. Secure, Efficient Data Transport and Replica Management for High-Performance Data-Intensive Computing, 2000. URL: http://www.globus.org.
• 3: Mark Allman, On the generation and use of TCP acknowledgments, ACM SIGCOMM Computer Communication Review, v.28 n.5, p.4-21, Oct. 1998 [doi>10.1145/303297.303301]
• 4: M. Allman, V. Paxson, and W. Stevens. TCP Congestion Control. RFC 2581, April 1999.
• 5: Mark Allman , Vern Paxson, On estimating end-to-end network path properties, Proceedings of the conference on Applications, technologies, architectures, and protocols for computer communication, p.263-274, August 30-September 03, 1999, Cambridge, Massachusetts, United States
• 6: P. Avery and I. Foster. The GriPhyN Project: Towards Petascale Virtual Data Grids, 2001. Technical Report GriPhyN-2001-15 URL: http://www.griphyn.org/.
• 7: D. Bansal and H. Balakrishnan. Binomial congestion control algorithms, 2001.
• 8: Wes Bethel , Brian Tierney , Jason lee , Dan Gunter , Stephen Lau, Using high-speed WANs and network data caches to enable remote and distributed visualization, Proceedings of the 2000 ACM/IEEE conference on Supercomputing (CDROM), p.28-es, November 04-10, 2000, Dallas, Texas, United States
• 9: R. Braden. Requirements for Internet Hosts -- Communication Layers. RFC 1122, October 1928.
• 10: Lawrence S. Brakmo , Sean W. O'Malley , Larry L. Peterson, TCP Vegas: new techniques for congestion detection and avoidance, Proceedings of the conference on Communications architectures, protocols and applications, p.24-35, August 31-September 02, 1994, London, United Kingdom
• 11: Lee Breslau , Deborah Estrin , Kevin Fall , Sally Floyd , John Heidemann , Ahmed Helmy , Polly Huang , Steven McCanne , Kannan Varadhan , Ya Xu , Haobo Yu, Advances in Network Simulation, Computer, v.33 n.5, p.59-67, May 2000 [doi>10.1109/2.841785]
• 12: A. Chervenak, I. Foster, C. Kesselman, C. Salisbury, and S. Tuecke. The data grid: Towards an architecture for the distributed management and analysis of large scientific datasets. In Network Storage Symposium (NetStore '99), 1999.
• 13: D.-M. Chiu , R. Jain, Analysis of the increase and decrease algorithms for congestion avoidance in computer networks, Computer Networks and ISDN Systems, v.17 n.1, p.1-14, June 10, 1989 [doi>10.1016/0169-7552(89)90019-6]
• 14: D. Clark, M. Lambert, and L. Zhang. NETBLT: A Bulk Data Transfer Protocol. RFC 998, March 1987.
• 15: R. L. Cottrell and C. Logg. A new high performance network and application monitoring infrastructure, 2002.
• 16: J. Crowcroft and P. Oechslin. Differentiated end-to-end internet services using a weighted proportional fair sharing tcp, 1998.
• 17: T. Dunigan and F. Fowler. A TCP-over-UDP test harness. Technical report, Oak Ridge National Laboratory, Oak Ridge, TN, 2002. ORNL/TM-2002/76.
• 18: EU. Eu datagrid project, 2001. URL: http://www.eu-datagrid.org/.
• 19: G. Farrache. "bbftp", 2000. URL: http://doc.in2p3.fr/bbftp/.
• 20: M. Fisk and W. Feng. Dynamic Right-Sizing in TCP. In Los Alamos Computer Science Institute Symposium, 2001.
• 21: S. Floyd. Congestion Control Principles. RFC 2914, September 2000.
• 22: S. Floyd. HighSpeed TCP for Large Congestion Windows. IETF draft, work in progress, May 2002. URL: http://www.icir.org/floyd/papers/draft-floyd-tcp-highspeed-00c.txt.
• 23: S. Floyd. Limited Slow-Start for TCP with Large Congestion Windows. IETF draft, work in progress, May 2002. URL: http://www.icir.org/floyd/papers/draft-floyd-tcp-slowstart-00b.txt.
• 24: Dan Gunter , Brian Tierney , Keith Jackson , Jason Lee , Martin Stoufer, Dynamic Monitoring of High-Performance Distributed Applications, Proceedings of the 11 th IEEE International Symposium on High Performance Distributed Computing HPDC-11 20002 (HPDC'02), p.163, July 24-26, 2002
• 25: Thomas J. Hacker , Brian D. Athey , Brian Noble, The End-to-End Performance Effects of Parallel TCP Sockets on a Lossy Wide-Area Network, Proceedings of the 16th International Parallel and Distributed Processing Symposium, p.314, April 15-19, 2002
• 26: A. Hanushevsky. "bbcp", 2000. URL: http://www.slac.stanford.edu/abh/bbcp/.
• 27: T. H. Henderson, E. Sahouria, S. McCanne, and R. H. Katz. On improving the fairness of TCP congestion avoidance. IEEE Globecomm conference, Sydney, 1998.
• 28: Internet2. Internet2 NetFlow: Weekly Reports, 2002. URL: http://netflow.internet2.edu/weekly/.
• 29: V. Jacobson, R. Braden, and D. Borman. RFC 1323: TCP extensions for high performance, May 1992.
• 30: K. Lahey. TCP problems with path MTU discovery. RFC 2923, September 2000.
• 31: Jason Lee , Dan Gunter , Martin Stoufer , Brian Tierney, Monitoring data archives for grid environments, Proceedings of the 2002 ACM/IEEE conference on Supercomputing, p.1-10, November 16, 2002, Baltimore, Maryland
• 32: J. Mahdavi and S. Floyd. TCP-Friendly Unicast Rate-Based Flow Control, 1997. URL: http://www.psc.edu/networking/papers/tcp_friendly.html.
• 33: M. Mathis. Pushing Up Performance for Everyone, 1999. URL: http://www.ncne.nlanr.net/news/workshop/1999/991205/Talks/mathis_991205_Pushing_Up_Performance/.
• 34: M. Mathis, R. Reddy, J. Heffner, and J. Saperia. TCP Extended Statistics MIB. IETF draft, work in progress, February 2002. URL: http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-tcp-mib-extension-00.txt.
• 35: Matt Mathis. "Web100", 2000. URL: http://www.web100.org.
• 36: K. Miller, K. Roberston, A. Tweedly, and M. White. StarBurst Multicast File Transfer Protocol (MFTP) Specification, 1998. URL: http://www.kblabs.com/lab/lib/drafts/draft-miller-mftp-spec-03.txt.html.
• 37: J. Mogul and S. Deering. Path MTU Discovery. RFC 1191, November 1990.
• 38: NIST. "NISTNet", 2001. URL: http://snad.ncsl.nist.gov/itg/nistnet/.
• 39: NLAR. "Auto-tuning FTP", 2000. URL: http://dast.nlanr.net/Features/Autobuf/.
• 40: NLAR. "iperf", 2000. URL: http://dast.nlanr.net/Projects/Iperf/.
• 41: V. Paxson, A. Adams, and M. Mathis. Experiences with nimi, 2000.
• 42: PPDG. Particle physics data grid, 2001. URL: http://www.ppdg.org/.
• 43: Jeffrey Semke , Jamshid Mahdavi , Matthew Mathis, Automatic TCP buffer tuning, Proceedings of the ACM SIGCOMM '98 conference on Applications, technologies, architectures, and protocols for computer communication, p.315-323, August 31-September 04, 1998, Vancouver, British Columbia, Canada
• 44: H. Sivakumar , S. Bailey , R. L. Grossman, PSockets: the case for application-level network striping for data intensive applications using high speed wide area networks, Proceedings of the 2000 ACM/IEEE conference on Supercomputing (CDROM), p.37-es, November 04-10, 2000, Dallas, Texas, United States
• 45: Peter Steenkiste, Adaptation Models for Network-Aware Distributed Computations, Proceedings of the Third International Workshop on Network-Based Parallel Computing: Communication, Architecture, and Applications, p.16-31, January 09, 1999
• 46: J. Stone and C. Partridge. When the CRC and TCP Checksum Disagree, 2000. URL: http://www.acm.org/sigcomm/sigcomm2000/conf/paper/sigcomm2000-9-1.pdf.
• 47: Brian L. Tierney , Dan Gunter , Jason Lee , Martin Stoufer , Joseph B. Evans, Enabling Network-Aware Applications, Proceedings of the 10th IEEE International Symposium on High Performance Distributed Computing (HPDC-10'01), p.281, August 07-09, 2001
• 48: Brian Tierney , William Johnston , Brian Crowley , Gary Hoo , Chris Brooks , Dan Gunter, The NetLogger Methodology for High Performance Distributed Systems Performance Analysis, Proceedings of the The Seventh IEEE International Symposium on High Performance Distributed Computing, p.260, July 28-31, 1998
• 49: Brian Tierney , Jason Lee , Ling Tony Chen , Hanan Herzog , Gary Hoo , Guojun Jin , William E. Johnston, Distributed parallel data storage systems: a scalable approach to high speed image servers, Proceedings of the second ACM international conference on Multimedia, p.399-405, October 15-20, 1994, San Francisco, California, United States [doi>10.1145/192593.192709]
• 50: Y. R Yang , Simon S. Lam, General AIMD Congestion Control, University of Texas at Austin, Austin, TX, 2000
• 51: Saul Youssef. "PacMAN", 2002. URL: http://physics.bu.edu/youssef/pacman/.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.6 Internetworking
          Subjects: Standards (e.g., TCP/IP)

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.2 Network Protocols

          Nouns: TCP/IP
      C.2.3 Network Operations
          Subjects: Network management
      C.2.5 Local and Wide-Area Networks
          Subjects: Internet (e.g., TCP/IP)

  D. Software
  D.4 OPERATING SYSTEMS
      D.4.4 Communications Management
          Subjects: Network communication

General Terms:
Design, Experimentation, Measurement, Performance

Keywords:
TCP, autotuning, data grids, high-performance networking