Architecture implications of high-speed I/O for distributed-memory computers

ABSTRACT

We consider the problem of high-speed I/O for a single application running on multiple nodes of a distributed-memory parallel computer. Our model is that the parallel system is connected to an I/O system that provides the interface between the internal connections of the parallel system and one or more external connections, such as HIPPI links. We identify two primary operations for this I/O system: scattering data from a high speed link across several lower speed links and gathering data from multiple links onto a single high speed link. We show that these core operations are the basis of the I/O system, independent of the relative speeds of the internal and external connections. We identify several architectural features that are critical for supporting high-speed scatter and gather operations. They include flexible routing methods in the parallel system, low overhead communication, and the ability to support multiple data streams in and out of the memory on the I/O node.

REFERENCES

• 1: Don Adams. Cray T3D system architecture overview. Cray Research Inc., September 1993. Revision 1.C.
• 2: Tom Blank. The Maspar MP-1 architecture. In IEEE Compcon Spring 1990, pages 20-24, San Francisco, February/March 1990. IEEE, IEEE Computer Society Press.
• 3: R. Bordawekar , J. M. del Rosario , A. Choudhary, Design and Evaluation of primitives for Parallel I/O, Proceedings of the 1993 ACM/IEEE conference on Supercomputing, p.452-461, December 1993, Portland, Oregon, United States [doi>10.1145/169627.169782]
• 4: S. Borkar , R. Cohn , G. Cox , S. Gleason , T. Gross, Warp: an integrated solution of high-speed parallel computing, Proceedings of the 1988 ACM/IEEE conference on Supercomputing, p.330-339, November 12-17, 1988, Orlando, Florida, United States
• 5: Shekhar Borkar , Robert Cohn , George Cox , Thomas Gross , H. T. Kung , Monica Lam , Margie Levine , Brian Moore , Wire Moore , Craig Peterson , Jim Susman , Jim Sutton , John Urbanski , Jon Webb, Supporting systolic and memory communication in iWarp, Proceedings of the 17th annual international symposium on Computer Architecture, p.70-81, May 28-31, 1990, Seattle, Washington, United States
• 6: Claudson Bornstein and Peter Steenkiste. Data reshuffling in support of fast i/o for distributed memory machines. In preparation, 1993.
• 7: Thinking Mahines Corporation. The Connection Machine CM-5 Technical Summary. Thinking Machines Corporation, 1991.
• 8: Erik DeBenedictis and Peter Madams. nCUBE's parallel I/O with UNIX compatibility. In Proceedings of the Sixth Distributed Memory Computing Conference, pages 270- 277. IEEE, April 1991.
• 9: Seema Hiranandani , Ken Kennedy , Chau-Wen Tseng, Compiling Fortran D for MIMD distributed-memory machines, Communications of the ACM, v.35 n.8, p.66-80, Aug. 1992 [doi>10.1145/135226.135230]
• 10: R.W. Hockney and Jesshope C.R. Parallel Computers. Adam Hilger Ltd., Bristol, U.K., 1981.
• 11: Intel Corporation. Paragon X/PS Product Overview, March 1991.
• 12: Sigurd L. Lillevik. The Touchstone 30 gigaflop Delta prototype. In Proceedings of the Sixth distribution Memory Computing Conference, pages 671-677. IEEE, April 1991.
• 13: Susan LoVeso, Marshall Isman, Andy Nanopoulos, William Nesheim, Ewan D. Milne, and Richard Wheeler. sfs: A parallel file system for the CM-5. In Proceedings of the Summer 1993 USENIX Conference, pages 291-305, Cincinnati, Ohio, June 1993. USENIX.
• 14: nCUBE Corp. nCUBE2: Technical Overview. nCUBE Corporation, Foster City, CA., 1992.
• 15: John R. Nickolls. The design of the Maspar MP-1: A cost effective massively parallel computer. In IEEE Compcon Spring 1990, pages 25-28, San Francisco, February/March 1990. IEEE Computer Society Press.
• 16: Paul Pierce. A concurrent file system for a highly parallel mass storage subsystem. In Proceedings of the Fourth Conference on Hypercubes, Concurrent Computers, and Applications, volume 1, pages 155-160, California, March 1989.
• 17: T.W. Pratt, J. C. French, R M. Dickens, and S. A. Janet Jr. A comparison of the architecture and performance of two parallel file systems. In Proceedings of the Fourth Conference on Hypercubes, Concurrent Computers, and Applications, volume 1, pages 161-166, California, March 1989.
• 18: P. Steenkiste , M. Hemy , T. Mummert , B. Zill, Architecture and evaluation of a high-speed networking subsystem for distributed-memory systems, Proceedings of the 21ST annual international symposium on Computer architecture, p.154-163, April 18-21, 1994, Chicago, Illinois, United States
• 19: Peter Steenkiste , Brian Zill , H. T. Kung , Steven Schlick , Jim Hughes , Bob Kowalski , John Mullaney, A Host Interface Architecture for High-Speed Networks, Proceedings of the IFIP TC6/WG6.4 Fourth International Conference on High Performance Networking IV, p.31-46, December 14-18, 1992
• 20: Lewis W. Tucker , George G. Robertson, Architecture and Applications of the Connection Machine, Computer, v.21 n.8, p.26-38, August 1988 [doi>10.1109/2.74]

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.1 PROCESSOR ARCHITECTURES

Additional Classification:
  B. Hardware
  B.4 INPUT/OUTPUT AND DATA COMMUNICATIONS

  C. Computer Systems Organization

  D. Software
  D.4 OPERATING SYSTEMS
      D.4.4 Communications Management
          Subjects: Input/output

General Terms:
Design, Measurement, Performance, Theory