Runtime optimization of vector operations on large scale SMP clusters

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback

Runtime optimization of vector operations on large scale SMP clusters

Full text

Pdf (1.34 MB)

Source

PACT archive
Proceedings of the 17th international conference on Parallel architectures and compilation techniques table of contents

Toronto, Ontario, Canada

SESSION: I/O optimizations table of contents

Pages 122-132

Year of Publication: 2008

ISBN:978-1-60558-282-5

Authors

Costin Iancu	Lawrence Berkeley National Laboratory, Berkeley, CA, USA
Steven Hofmeyr	Lawrence Berkeley National Laboratory, Berkeley, CA, USA

Sponsors

ACM: Association for Computing Machinery
SIGARCH: ACM Special Interest Group on Computer Architecture

Publisher

ACM New York, NY, USA

Bibliometrics

Downloads (6 Weeks): 6, Downloads (12 Months): 82, Citation Count: 0

Additional Information:

abstract references index terms collaborative colleagues

Tools and Actions:	Request Permissions Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

DOI Bookmark:	Use this link to bookmark this Article: http://doi.acm.org/10.1145/1454115.1454134 What is a DOI?

ABSTRACT

"Vector" style communication operations transfer multiple disjoint memory regions within one logical step. These operations are widely used in applications, they do improve application performance, and their behavior has been studied and optimized using different implementation techniques across a large variety of systems. In this paper we present a methodology for the selection of the best performing implementation of a vector operation from multiple alternative implementations. Our approach is designed to work for systems with wide SMP nodes where we believe that most published studies fail to correctly predict performance. Due to the emergence of multi-core processors we believe that techniques similar to ours will be incorporated for performance reasons in communication libraries or language runtimes.

The methodology relies on the exploration of the application space and a classification of the regions within this space where a particular implementation method performs best. We use micro-benchmarks to measure the performance of an implementation for a given point in the application space and then compose profiles that compare the performance of two given implementations. These profiles capture an empirical upper bound for the performance degradation of a given protocol under heavy node load. At runtime, the application selects the implementation according to these performance profiles. Our approach provides performance portability and using our dynamic multi-protocol selection we have been able to improve the performance of a NAS Parallel Benchmarks workload by 22% on an IBM large scale cluster. Very positive results have also been obtained on large scale InfiniBand and Cray XT systems. This work indicates that perhaps the most important factor for application performance on wide SMP systems is the successful management of load on the Network Interface Cards.

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	Albert Alexandrov , Mihai F. Ionescu , Klaus E. Schauser , Chris Scheiman, LogGP: incorporating long messages into the LogP model for parallel computation, Journal of Parallel and Distributed Computing, v.44 n.1, p.71-79, July 10, 1997 [doi>10.1006/jpdc.1997.1346]
	2	Bassi IBM p575 POWER5. LBNL National Energy Research Supercomputing Center.
	3	Bigben Cray XT3 MPP. Pittsburgh Supercomputing Center.
	4	C. Bell, D. Bonachea, R. Nishtala, and K. Yelick. Optimizing Bandwidth Limited Problems Using One-Sided Communication and Overlap. Proceedings of the 20th International Parallel and Distributed Processing Symposium (IPDPS), 2006.
	5	Dan Bonachea, GASNet Specification, v1.1, University of California at Berkeley, Berkeley, CA, 2002
	6	D. Bonachea. Proposal for Extending the UPC Memory Copy Library Functions and Supporting Extensions to GASNet, v1.0. Technical Report LBNL-56495, Lawrence Berkeley National Laboratory, 2004.
	7	S. Byna, W. D. Gropp, X.-H. Sun, and R. Thakur. Improving the Performance of MPI Derived Datatypes by Optimizing Memory-Access Cost. In IEEE International Conference on Cluster Computing, 2003.
	8	Co-Array Fortran - Technical Specification. Available at http://www.co-array.org/caf_def.htm.
	9	Kirk W. Cameron , Rong Ge , Xian-He Sun, $\log_{\rm n}{\rm P}$ and $\log_{3}{\rm P}$: Accurate Analytical Models of Point-to-Point Communication in Distributed Systems, IEEE Transactions on Computers, v.56 n.3, p.314-327, March 2007 [doi>10.1109/TC.2007.38]
	10	Yuri Dotsenko , Cristian Coarfa , John Mellor-Crummey, A Multi-Platform Co-Array Fortran Compiler, Proceedings of the 13th International Conference on Parallel Architectures and Compilation Techniques, p.29-40, September 29-October 03, 2004 [doi>10.1109/PACT.2004.3]
	11	Costin Iancu , Wei Chen , Katherine Yelick, Performance portable optimizations for loops containing communication operations, Proceedings of the 22nd annual international conference on Supercomputing, June 07-12, 2008, Island of Kos, Greece [doi>10.1145/1375527.1375567]
	12	Costin C. Iancu , Erich Strohmaier, Optimizing communication overlap for high-speed networks, Proceedings of the 12th ACM SIGPLAN symposium on Principles and practice of parallel programming, March 14-17, 2007, San Jose, California, USA [doi>10.1145/1229428.1229436]
	13	Jacquard AMD Opteron cluster. LBNL National Energy Research Supercomputing Center.
	14	N. Koziris , A. Sotiropoulos , G. Goumas, A pipelined schedule to minimize completion time for loop tiling with computation and communication overlapping, Journal of Parallel and Distributed Computing, v.63 n.11, p.1138-1151, November 2003 [doi>10.1016/S0743-7315(03)00102-3]
	15	M. Krishnan and J. Nieplocha. Optimizing Performance on Linux Clusters Using Advanced Communication Protocols: Achieving Over 10 Teraflops on a 8.6 Teraflops Linpack-Rated Linux Cluster. Proceedings of the 6th International Conference on Linux clusters: The HPC Revolution, 2005.
	16	The NAS Parallel Benchmarks. Available at http://www.nas.nasa.gov/Software/NPB.
	17	Jarek Nieplocha , Bryan Carpenter, ARMCI: A Portable Remote Memory Copy Libray for Ditributed Array Libraries and Compiler Run-Time Systems, Proceedings of the 11 IPPS/SPDP'99 Workshops Held in Conjunction with the 13th International Parallel Processing Symposium and 10th Symposium on Parallel and Distributed Processing, p.533-546, April 12-16, 1999
	18	Jaroslaw Nieplocha , Robert J. Harrison , Richard J. Littlefield, Global arrays: a nonuniform memory access programming model for high-performance computers, The Journal of Supercomputing, v.10 n.2, p.169-189, June 1996
	19	Jarek Nieplocha , Vinod Tipparaju , Manoj Krishnan, Optimizing Strided Remote Memory Access Operations on the Quadrics QsNetII Network Interconnect, Proceedings of the Eighth International Conference on High-Performance Computing in Asia-Pacific Region, p.28, November 30-December 03, 2005 [doi>10.1109/HPCASIA.2005.62]
	20	R. Numrich and J. Reid. Co-Array Fortran for Parallel Programming. Technical Report RAL-TR-1998-060, Rutherford Appleton Laboratory, 1998.
	21	Ranger SUN Constellation linux Cluster. Texas Advanced Computing Center, University of Texas at Austin.
	22	G. Santhanaraman, D. Wu, and D. K. Panda. Zero-Copy MPI Derived Datatype Communication over InfiniBand. In Recent Advances in Parallel Virtual Machine and Message Passing Interface, 11th European PVM/MPI Users' Group Meeting, 2004.
	23	Jimmy Su , Katherine Yelick, Automatic Support for Irregular Computations in a High-Level Language, Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05) - Papers, p.53.2, April 04-08, 2005 [doi>10.1109/IPDPS.2005.118]
	24	Nathan Thomas , Gabriel Tanase , Olga Tkachyshyn , Jack Perdue , Nancy M. Amato , Lawrence Rauchwerger, A framework for adaptive algorithm selection in STAPL, Proceedings of the tenth ACM SIGPLAN symposium on Principles and practice of parallel programming, June 15-17, 2005, Chicago, IL, USA [doi>10.1145/1065944.1065981]
	25	V. Tipparaju, M. Krishnan, J. Nieplocha, and D. P. G. Santhanaraman. Exploiting Non-blocking Remote Memory Access Communication in Scientific Benchmarks. Proceedings of the 2003 International Conference on High Performance Computng, HiPC'2003, 2003.
	26	V. Tipparaju, G. Santhanaraman, J. Nieplocha, and D. K. Panda. Host-Assisted Zero-Copy Remote Memory Access Communication on InfiniBand. In 18th International Parallel and Distributed Processing Symposium, 2004.
	27	Keith D. Underwood , Michael J. Levenhagen , Ron Brightwell, Evaluating NIC hardware requirements to achieve high message rate PGAS support on multi-core processors, Proceedings of the 2007 ACM/IEEE conference on Supercomputing, November 10-16, 2007, Reno, Nevada [doi>10.1145/1362622.1362671]
	28	UPC Language Specification, Version 1.0. Available at http://upc.gwu.edu.
	29	Jeffrey S. Vetter , Frank Mueller, Communication Characteristics of Large-Scale Scientific Applications for Contemporary Cluster Architectures, Proceedings of the 16th International Parallel and Distributed Processing Symposium, p.96, April 15-19, 2002
	30	Thorsten von Eicken , David E. Culler , Seth Copen Goldstein , Klaus Erik Schauser, Active messages: a mechanism for integrated communication and computation, Proceedings of the 19th annual international symposium on Computer architecture, p.256-266, May 19-21, 1992, Queensland, Australia
	31	K. Yelick, L. Semenzato, G. Pike, C. Miyamoto, B. Liblit, A. Krishnamurthy, P. Hilfinger, S. Graham, D. Gay, P. Colella, and A. Aiken. Titanium: A High-Performance Java Dialect. In Proceedings of the ACM 1998 Workshop on Java for High-Performance Network Computing. ACM Press, 1998.