Early Evaluation of the Cray X1

ABSTRACT

Oak Ridge National Laboratory installed a 32 processor Cray X1 in March, 2003, and will have a 256 processor system installed by October, 2003. In this paper we describe our initial evaluation of the X1 architecture, focusing on microbenchmarks, kernels, and application codes that highlight the performance characteristics of the X1 architecture and indicate how to use the system most efficiently.

REFERENCES

• 1: [1] D. H. BAILEY, T. HARRIS, R. F. V. DER WIJNGAART, W. SAPHIR, A. WOO, AND M. YARROW, The NAS Parallel Benchmarks 2.0, Tech. Rep. NAS-95-010, NASA Ames Research Center, Moffett Field, CA, 1995.
• 2: [2] M. B. BLACKMON, B. BOVILLE, F. BRYAN, R. DICKINSON, P. GENT, J. KIEHL, R. MORITZ, D. RANDALL, J. SHUKLA, S. SOLOMON, G. BONAN, S. DONEY, I. FUNG, J. HACK, E. HUNKE, AND J. HURREL, The Community Climate System Model, BAMS, 82 (2001), pp. 2357-2376.
• 3: [3] A. S. BLAND, J. J. DONGARRA, J. B. DRAKE, T. H. DUNIGAN, JR., T. H. DUNNING, JR., G. A. GEIST, B. GORDA, W. D. GROPP, R. J. HARRISON, R. KENDALL, D. KEYES, J. A. NICHOLS, L. OLIKER, H. SIMON, R. STEVENS, J. B. WHITE III, P. H. WORLEY, AND T. ZACHARIA, Cray X1 Evaluation, Tech. Rep. ORNL/TM-2003/67, Oak Ridge National Laboratory, Oak Ridge, TN, March 2003.
• 4: J. Candy , R. E. Waltz, An Eulerian gyrokinetic-Maxwell solver, Journal of Computational Physics, v.186 n.2, p.545-581, 10 April 2003 [doi>10.1016/S00021-9991(03)00079-2]
• 5: [5] W. W. CARLSON, J. M. DRAPER, D. E. CULLER, K. YELICK, E. BROOKS, AND K. WARREN, Introduction to UPC and language specification, Technical Report CCS-TR-99-157, Center for Computing Sciences, 17100 Science Dr., Bowie, MD 20715, May 1999.
• 6: [6] CRAY INC., Cray X1. http://www.cray.com/products/systems/x1/.
• 7: J. J. Dongarra , Jeremy Du Croz , Sven Hammarling , I. S. Duff, A set of level 3 basic linear algebra subprograms, ACM Transactions on Mathematical Software (TOMS), v.16 n.1, p.1-17, March 1990 [doi>10.1145/77626.79170]
• 8: [8] K. FEIND, Shared Memory Access (SHMEM) Routines, in CUG 1995 Spring Proceedings, R. Winget and K. Winget, ed., Eagen, MN, 1995, Cray User Group, Inc., pp. 303-308.
• 9: Roger Hockney , Michael Berry, Public international benchmarks for parallel computers: PARKBENCH committee: Report-1, Scientific Programming, v.3 n.2, p.100-146, Summer 1994
• 10: [10] P. W. JONES, The Los Alamos Parallel Ocean Program (POP) and coupled model on MPP and clustered SMP computers, in Making its Mark - The Use of Parallel Processors in Meteorology: Proceedings of the Seventh ECMWF Workshop on Use of Parallel Processors in Meteorology, G.-R. Hoffman and N. Kreitz, eds., World Scientific Publishing Co. Pte. Ltd., Singapore, 1999.
• 11: [11] T. A. MAIER, M. JARRELL, J. HAGUE, J. B. WHITE III, AND T. C. SCHULTHESS, Decisive change in the understanding of high-temperature superconductivity through improved computing capability, 2003. DOE MICS Highlight.
• 12: [12] J. D. MCCALPIN, Memory Bandwidth and Machine Balance in Current High Performance Computers , IEEE Computer Society Technical Committee on Computer Architecture Newsletter, (1995). http://tab.computer.org/tcca/news/dec95/dec95.htm.
• 13: [13] MPI Committee, MPI: a message-passing interface standard, Internat. J. Supercomputer Applications, 8 (1994), pp. 165-416.
• 14: Jaroslaw Nieplocha , Robert J. Harrison , Richard J. Littlefield, Global arrays: a portable "shared-memory" programming model for distributed memory computers, Proceedings of the 1994 ACM/IEEE conference on Supercomputing, November 14-18, 1994, Washington, D.C. [doi>10.1145/602770.602833]
• 15: Robert W. Numrich , John Reid, Co-array Fortran for parallel programming, ACM SIGPLAN Fortran Forum, v.17 n.2, p.1-31, Aug. 1998 [doi>10.1145/289918.289920]
• 16: [16] OpenMP ARCHITECTURE REVIEW BOARD, OpenMP: A proposed standard api for shared memory programming . (available from http://www.openmp.org/openmp/mp-documents/paper/paper.ps), October 1997.
• 17: James R. Taft, Achieving 60 GFLOP/s on the production CFD code OverFLow-MLP, Parallel Computing, v.27 n.4, p.521-536, March 1, 2001 [doi>10.1016/S0167-8191(00)00072-7]
• 18: [18] A. J. VAN DER STEEN, The benchmark of the EuroBen group, Parallel Computing, 17 (1991), pp. 1211- 1221.
• 19: [19] A. J. WALLCRAFT, SPMD OpenMP vs MPI for Ocean Models, in Proceedings of the First European Workshop on OpenMP, Lund, Sweden, 1999, Lund University. http://www.it.lth.se/ewomp99.
• 20: [20] P. H. WORLEY AND I. T. FOSTER, PSTSWM: a parallel algorithm testbed and benchmark code for spectral general circulation models, Tech. Rep. ORNL/TM-12393, Oak Ridge National Laboratory, Oak Ridge, TN, (in preparation).
• 21: [21] P. H. WORLEY AND B. TOONEN, A users' guide to PSTSWM, Tech. Rep. ORNL/TM-12779, Oak Ridge National Laboratory, Oak Ridge, TN, July 1995.