Aspects of GPU for general purpose high performance computing

ABSTRACT

We discuss hardware and software aspects of GPGPU, specifically focusing on NVIDIA cards and CUDA, from the viewpoints of parallel computing. The major weak points of GPU against newest supercomputers are identified to be and summarized as only four points: large SIMD vector length, small memory, absence of fast L2 cache, and high register spill penalty. As software concerns, we derive optimal scheduling algorithm for latency hiding of host-device data transfer, and discuss SPMD parallelism on GPUs.

REFERENCES

• 1: T. Aoki and S. Ogawa, "Iterative Solver using CUDA for Poisson Equation," Proceedings of 36th Visualization Symposium, vol. 28, suppl. no. 1, pp. 255--258, 2008 (in Japanese).
• 2: Akira Nukada , Yasuhiko Ogata , Toshio Endo , Satoshi Matsuoka, Bandwidth intensive 3-D FFT kernel for GPUs using CUDA, Proceedings of the 2008 ACM/IEEE conference on Supercomputing, November 15-21, 2008, Austin, Texas
• 3: S. Hirasawa, Y. Nakanishi, H. Watanabe, and H. Honda, "Common Description Language of SIMD Instructions for Performance Portability," Proceedings of the 2008 International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA'08), pp. 52--58, 2008.
• 4: NVIDIA, "NVIDIA CUDA Compute Unified Device Architecture Programming Guide," version 2.0, 2008.
• 5: The 31st TOP500 list, http://www.top500.org/, November, 2008.
• 6: NVIDIA, "The CUDA Compiler Driver NVCC," version 2.0, 2008.
• 7: Vasily Volkov , James W. Demmel, Benchmarking GPUs to tune dense linear algebra, Proceedings of the 2008 ACM/IEEE conference on Supercomputing, November 15-21, 2008, Austin, Texas
• 8: Veeravalli Bharadwaj , Debasish Ghose , Thomas G. Robertazzi, Divisible Load Theory: A New Paradigm for Load Scheduling in Distributed Systems, Cluster Computing, v.6 n.1, p.7-17, January 2003 [doi>10.1023/A:1020958815308]