Communication in the KSR1 MPP: performance evaluation using synthetic workload experiments

ABSTRACT

We have developed an automatic technique for evaluating the communication performance of massively parallel processors (MPPs). Both communication latency and the amount of communication are investigated as a function of a few basic parameters that characterize an application workload. Parameter values are captured in an automatically generated sparse matrix that multiplies a dense vector in the synthetic workload. Our approach is capable of explaining the degradation of processor performance caused by communication. Using the Kendall Square Research KSR1 MPP as a case study, we demonstrate the effectiveness of the technique through a series of experiments used to characterize the communication performance. We show that read and write communciation latencies vary from 150 to 180 and from 80 to 100 processor cycles, respectively. We show that the read communication latency approximates a linear function of the total system communciation (in subpages), write communication approximates a linear function of the number of distinct shared subpages, and that KSR's automatic update feature is effective in reducing the number of read communications given careful binding of threads to processors.

REFERENCES

• 1: Eric L. Boyd , Edward S. Davidson, Hierarchical performance modeling with MACS: a case study of the convex C-240, Proceedings of the 20th annual international symposium on Computer architecture, p.203-210, May 16-19, 1993, San Diego, California, United States
• 2: J. H. Tang , E. S. Davidson , J. Tong, Polycyclic Vector scheduling vs. Chaining on 1-Port Vector supercomputers, Proceedings of the 1988 ACM/IEEE conference on Supercomputing, p.122-122, November 12-17, 1988, Orlando, Florida, United States
• 3: William Mangione-Smith , Santosh G. Abraham , Edward S. Davidson, A Performance Comparison of the IBM RS/6000 and the Astronautics ZS-1, Computer, v.24 n.1, p.39-46, January 1991 [doi>10.1109/2.67192]
• 4: W.H. Mangione-Smith, T-P. Shih, S. G. Abraham, E. S. Davidson, "Approaching a Machine-Application Bound in Delivered Performance on Scientific Code," IEEE Proceedings, August, 1993, pp. 1166-1178.
• 5: W. Azeem. "Modeling and Approaching the Deliverable Performance Capability of the KSR1 Processor," University of Michigan, Technical Report, CSE-TR-164-93, June, 1993.
• 6: E.L. Boyd, W. Azeem, H-H. Lee, T-P. Shih, S-H. Hung, E. S. Davidson. "A Hierarchial Approach to Modeling and Improving the Performance of Scientific Applications on the KSR1," to apper in the Proceedings of the 1994 International Conference on Parallel Processing.
• 7: Eric L. Boyd , John-David Wellman , Santosh G. Abraham , Edward S. Davidson, Evaluating the communication performance of MPPs using synthetic sparse matrix multiplication workloads, Proceedings of the 7th international conference on Supercomputing, p.240-250, July 19-23, 1993, Tokyo, Japan [doi>10.1145/165939.165974]
• 8: R. H. Saavedra , R. S. Gaines , M. J. Carlton, Micro benchmark analysis of the KSR1, Proceedings of the 1993 ACM/IEEE conference on Supercomputing, p.202-213, December 1993, Portland, Oregon, United States [doi>10.1145/169627.169695]
• 9: K. Harzallah, H. Li, K. Sevcik. "Evaluating the Effect of the Auto-Update on the Kendall Square KSR1," Computer Systems Research Institute University of Toronto Technical Report CSRI-291, October, 1993.
• 10: T.D. Wagner, E. Smirni, A. W. Apon, M. Madhukar, L. W. Dowdy. "Measuring the Effects of Thread Placement on the Kendall Square KSR1," Oak Ridge National Laboratory Technical Report ORNL/TM-12462, August, 1993.
• 11: Per Stenström , Truman Joe , Anoop Gupta, Comparative performance evaluation of cache-coherent NUMA and COMA architectures, Proceedings of the 19th annual international symposium on Computer architecture, p.80-91, May 19-21, 1992, Queensland, Australia
• 12: J. P. Singh , T. Joe , J. L. Hennessy , A. Gupta, An empirical comparison of the Kendall Square Research KSR-1 and Stanford DASH multiprocessors, Proceedings of the 1993 ACM/IEEE conference on Supercomputing, p.214-225, December 1993, Portland, Oregon, United States [doi>10.1145/169627.169699]
• 13: KSR1 Principles of Operation, Kendall Square Research Corporation, Waltham, MA, 1992.
• 14: KSR1 Technical Summary, Kendall Square Research Corporation, Waltham, MA, 1992.
• 15: Erik Hagersten , Anders Landin , Seif Haridi, DDM: A Cache-Only Memory Architecture, Computer, v.25 n.9, p.44-54, September 1992 [doi>10.1109/2.156381]
• 16: T. H. Dunign. "Kendall Square Mdtiprocessor: Early experiences and performance," Oak Ridge National Laboratory Technical Report ORNL/TM-12065, April, 1992.
• 17: D. Windheiser, E. L. Boyd, E. Hao, S. G. Abraham, E. S. Davidson. "KRS1 Multiprocessor, Analysis of Latency Hiding Techniques in a Sparse Solver," Proceedings of the 7th International Parallel Processing Symposium, April, 1993, pp. 454-461.
• 18: Bassam Z. Kahhaleh, Analysis of Memory Latency Factors and Their Impact on KSR1 Performance, Proceedings of the 8th International Symposium on Parallel Processing, p.649-656, April 01, 1994
• 19: E. Rosti , E. Smirni , T. D. Wagner , A. W. Apon , L. W. Dowdy, The KSR1: experimentation and modeling of poststore, Proceedings of the 1993 ACM SIGMETRICS conference on Measurement and modeling of computer systems, p.74-85, May 10-14, 1993, Santa Clara, California, United States

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.1 PROCESSOR ARCHITECTURES

Additional Classification:
  C. Computer Systems Organization

General Terms:
Design, Experimentation, Measurement, Performance, Theory