Separated high-bandwidth and low-latency communication in the cluster interconnect Clint

ABSTRACT

An interconnect for a high-performance cluster has to be optimized in respect to both high throughput and low latency. To avoid the tradeoff between throughput and latency, the cluster interconnect Clint has a segregated architecture that provides two physically separate transmission channels: A bulk channel optimized for high-bandwidth traffic and a quick channel optimized for low-latency traffic. Different scheduling strategies are applied. The bulk channel uses a scheduler that globally allocates time slots on the transmission paths before packets are sent off. This way collisions as well as blockages are avoided. In contrast, the quick channel takes a best-effort approach by sending packets whenever they are available thereby risking collisions and retransmissions.Simulation results clearly show the performance advantages of the segregated architecture. The carefully scheduled bulk channel can be loaded nearly to its full capacity without exhibiting head-of-line blocking that limits many networks while the quick channel provides low-latency communication even in the presence of high-bandwidth traffic.

REFERENCES

• 1: Nanette J. Boden , Danny Cohen , Robert E. Felderman , Alan E. Kulawik , Charles L. Seitz , Jakov N. Seizovic , Wen-King Su, Myrinet: A Gigabit-per-Second Local Area Network, IEEE Micro, v.15 n.1, p.29-36, February 1995 [doi>10.1109/40.342015]
• 2: Brent N. Chun , Alan M. Mainwaring , David E. Culler, Virtual Network Transport Protocols for Myrinet, IEEE Micro, v.18 n.1, p.53-63, January 1998 [doi>10.1109/40.653035]
• 3: William J. Dally, Virtual-channel flow control, Proceedings of the 17th annual international symposium on Computer Architecture, p.60-68, May 28-31, 1990, Seattle, Washington, United States
• 4: J. Duato, P. López, F. Silla: A High Performance Router Architecture for Interconnection Networks. Proc. Int. Conf. On Parallel Processing. 1996.
• 5: Davib B. Gustavson, The Scalable Coherent Interface and Related Standards Projects, IEEE Micro, v.12 n.1, p.10-22, January 1992 [doi>10.1109/40.124376]
• 6: Nils Gura , Hans Eberle, The Least Choice First Scheduling Method for High-Speed Network Switche, Proceedings of the 16th International Parallel and Distributed Processing Symposium, p.316, April 15-19, 2002
• 7: J. Hoffman: HIPPI-6400 Technology Dissemination. Proc. of SPIE, vol. 2917, 1996, pp. 442--430.
• 8: R. Horst, D. Garcia: Servernet SAN I/O Architecture. Hot Interconnects V Symposium, Stanford, Aug. 21--23, 1997.
• 9: M. Karol, M. Hluchyi, S. Morgan: Input versus Output Queuing on a Space-Division Packet Switch. IEEE Transactions on Communications, C-35(12):1347--1356, December 1987.
• 10: R. Kessler, J. Schwarzmeier: Cray T3D: A New Dimension for Cray Research. Proc. 38th IEEE Int. Computer Conf., 1993, pp. 176--182.
• 11: Jun Seong Kim , David J. Lilja, Utilizing Heterogeneous Networks in Distributed Parallel Computing Systems, Proceedings of the 6th International Symposium on High Performance Distributed Computing (HPDC '97), p.336, August 05-08, 1997
• 12: JunSeong Kim , David J. Lilja, Performance-Based Path Determination for Interprocessor Communication in Distributed Computing Systems, IEEE Transactions on Parallel and Distributed Systems, v.10 n.3, p.316-327, March 1999 [doi>10.1109/71.755832]
• 13: Charles E. Leiserson , Zahi S. Abuhamdeh , David C. Douglas , Carl R. Feynman , Mahesh N. Ganmukhi , Jeffrey V. Hill , Daniel Hillis , Bradley C. Kuszmaul , Margaret A. St. Pierre , David S. Wells , Monica C. Wong , Shaw-Wen Yang , Robert Zak, The network architecture of the Connection Machine CM-5 (extended abstract), Proceedings of the fourth annual ACM symposium on Parallel algorithms and architectures, p.272-285, June 29-July 01, 1992, San Diego, California, United States [doi>10.1145/140901.141883]
• 14: Alan Mainwaring , David Culler, Active Message Applications Programming Interface, University of California at Berkeley, Berkeley, CA, 1996
• 15: Alan M. Mainwaring , David E. Culler, Design challenges of virtual networks: fast, general-purpose communication, Proceedings of the seventh ACM SIGPLAN symposium on Principles and practice of parallel programming, p.119-130, May 04-06, 1999, Atlanta, Georgia, United States
• 16: N. McKeown, C. Calamvokis, S. Chuang: A 2.5 Tb/s LCS Switch Core. Hot Chips 13, August 19--21 2001, Stanford, California.
• 17: Message P Forum, MPI: A Message-Passing Interface Standard, University of Tennessee, Knoxville, TN, 1994
• 18: Scott Pakin , Vijay Karamcheti , Andrew A. Chien, Fast Messages: Efficient, Portable Communication for Workstation Clusters and MPPs, IEEE Parallel & Distributed Technology: Systems & Technology, v.5 n.2, p.60-73, April 1997
• 19: National Committee for Information Technology Standardization: Scheduled Transfer Protocol (ST). Task Group btgt11.1, rev. 3.6, January 31, 2000, www.hippi.org.
• 20: V. S. Sunderam, PVM: a framework for parallel distributed computing, Concurrency: Practice and Experience, v.2 n.4, p.315-339, Dec. 1990 [doi>10.1002/cpe.4330020404]

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.4 Communications Management
          Subjects: Network communication

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.1 Network Architecture and Design
          Subjects: Network communications
      C.2.3 Network Operations
          Subjects: Network monitoring

  E. Data
  E.4 CODING AND INFORMATION THEORY
      Subjects: Formal models of communication

General Terms:
Experimentation, Management, Measurement, Performance