Parallel programming: can we PLEASE get it right this time?

ABSTRACT

The computer industry has a problem. As Moore's law marches on, it will be exploited to double cores, not frequencies. But all those cores, growing to 8, 16 and beyond over the next several years, are of little value without parallel software. Where will this come from? With few exceptions, only graduate students and other strange people write parallel software. Even for numerically intensive applications, where parallel algorithms are well understood, professional software engineers almost never write parallel software.

Somehow we need to (1) design many core systems programmers can actually use and (2) provide programmers with parallel programming environments that work. The good news is we have 25+ years of history in the HPC space to guide us. The bad news is that few people are paying attention to this experience.

This talk looks at the history of parallel computing to develop a set of anecdotal rules to follow as we create manycore systems and their programming environments. A common theme is that just about every mistake we could make has already been made by someone. So rather than reinvent these mistakes, let's learn from the past and "do it right this time".

REFERENCES

• 1: Zhao-Hui Du , Chu-Cheow Lim , Xiao-Feng Li , Chen Yang , Qingyu Zhao , Tin-Fook Ngai, A cost-driven compilation framework for speculative parallelization of sequential programs, Proceedings of the ACM SIGPLAN 2004 conference on Programming language design and implementation, June 09-11, 2004, Washington DC, USA
• 2: Iyengar, Sheena S., & Lepper, Mark. "When choice is demotivating: Can one desire too much of a good thing?", Journal of Personality and Social Psychology 76, 2000, pp. 995--1006.
• 3: Iyengar, Sethi-S., Huberman, G., and Jiang, W. "How Much Choice is Too Much? Contributions to 401(k) Retirement Plans", in O. S. Mitchell & S. Utkus, (Eds.), Pension Design and Structure: New Lessons from Behavioral Finance, Oxford University Press, 2004, pp. 83--95
• 4: OpenMP specification, www.openmp.org
• 5: HPF, http://hpff.rice.edu
• 6: HPF Japanese association of High Performance Fortran, http://www.tokyo.rist.or.jp/jahpf/
• 7: MPI, http://www.mpi-forum.org/docs
• 8: T. Mattson, B. Sanders, B. Massingill, Patterns for Parallel Programming, Addison-Wesley Professional, 2004
• 9: T. R. G. Green and M. Petre, "Usability Analysis of Visual Programming Languages: a 'cognitive dimensions' framework", J. Visual Languages and Computing 7, pp 131--174, 1996
• 10: Steven Clarke, "Evaluating a new programming language", Proceedings of the 13^th Workshop of the Psychology of Programming Interest Group, Bournemouth UK, G. Kadoda (Ed.), 2001, pp. 275--289.
• 11: Christian Bienia, Sanjeev Kumar, Jaswinder Pal Singh and Kai Li, "The PARSEC Benchmark Suite: Characterization and Architectural Implications", Princeton University Technical Report TR-811-08, January 2008, ftp://ftp.cs.princeton.edu/techreports/2008/811.pdf
• 12: The Landscape of Parallel Computing Research: A View from Berkeley, http://view.eecs.berkeley.edu

INDEX TERMS

Primary Classification:
  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.3 Concurrent Programming
          Subjects: Parallel programming

General Terms:
Algorithms, Design, Experimentation, Human Factors, Languages, Performance

Keywords:
design patterns, parallel computing