A case study in programming for parallel-processors

ABSTRACT

An affirmative partial answer is provided to the question of whether it is possible to program parallel-processor computing systems to efficiently decrease execution time for useful problems. Parallel-processor systems are multiprocessor systems in which several of the processors can simultaneously execute separate tasks of a single job, thus cooperating to decrease the solution time of a computational problem. The processors have independent instruction counters, meaning that each processor executes its own task program relatively independently of the other processors. Communication between cooperating processors is by means of data in storage shared by all processors. A program for the determination of the distribution of current in an electrical network was written for a parallel-processor computing system, and execution of this program was simulated. The data gathered from simulation runs demonstrate the efficient solution of this problem, typical of a large class of important problems. It is shown that, with proper programming, solution time when NP processors are applied approaches 1/NP times the solution time for a single processor, while improper programming can actually lead to an increase of solution time with the number of processors. Storage interference and other measures of performance are discussed. Stability of the method of solution was also investigated.

REFERENCES

• 1: LEHMAN, M. A survey of problems and preliminary results concerning parallel processing and parallel processors. Proc. IEEE (Dec. 1966), 1889-1901.
• 2: ROSENFELD, JACK L., AND DRISCOLL, GRAHAM C. Solution of the Diriehlet problem on a simulated parallel processing system. Information Processing 68, paper C24, final Proc. IFIP 1968 Congress, North-Holland Pub. Co., Amsterdam (to be published).
• 3: DRISCOLL, GRAHAM C. Simulation of the Monte Carlo method on SIMP II. IBM Res. Rep. RC-2182, Thomas J. Watson Res. Cent., Yorktown Heights, N. Y., Aug. 1968.
• 4: KEELEY, J. F., ET AL. An application-oriented multiprocessing system. IBM Syst. J . 6, 2 (1967), 78-132.
• 5: ANDERSON, J. P., ET AL. D825-a multiple computer system for command and control. Proc. AFIPS 1962 Fall Joint Comp. Conf., Spartan Books, New York, pp. 86-96.
• 6: GIBSON, CHARLES T. Time sharing in the IBM System/360: Model 67. Proc. AFIPS 1966 Spring Joint Comput. Conf. Vol. 28, Spartan Books, New York, pp. 61-78.
• 7: BARNES, GEORGE H., ET AL. The ILLIAC IV computer. IEEE Trans. EC (Aug. 19668), 746-757.
• 8: POMERENE, J. H. An approach to parallel processing. Proc IFIP 1965 Congress Vol. 2, Spartan Books, New York, p. 322.
• 9: CONWAY, MELVIN E. A multiprocessor system design. Proc. AFIPS 1963 Fall Joint Comput. Conf., Spartan Books, New York, pp. 139-146.
• 10: Jack B. Dennis , Earl C. Van Horn, Programming semantics for multiprogrammed computations, Communications of the ACM, v.9 n.3, p.143-155, March 1966 [doi>10.1145/365230.365252]
• 11: TODD, JOHN (Ed.) Survey of Numerical Analysis. McGraw- Hill, New York, 1962.
• 12: VARGA, RICHARD S. Matrix Iterative Analysis. Prentice-Hall, Englewood Cliffs, N. J., 1962, pp. 56-80.
• 13: CHAZAN, D., AND MIRANKER, W. Chaotic relaxation. IBM Res. Rep. RC-1976, Thomas J. Watson Res. Cent., Yorktown Heights, N. Y., Jan. 1968 (accepted for publication in J. Linear Algebra).

INDEX TERMS

Primary Classification:
  F. Theory of Computation
  F.1 COMPUTATION BY ABSTRACT DEVICES
      F.1.2 Modes of Computation
          Subjects: Parallelism and concurrency

Additional Classification:
  D. Software
  D.1 PROGRAMMING TECHNIQUES
  D.4 OPERATING SYSTEMS
      D.4.1 Process Management
          Subjects: Multiprocessing/multiprogramming/multitasking

General Terms:
Design, Theory

Keywords:
Gauss-Seidel, Jacobi, convergence, electrical network, multiprocessor, multiprogramming, parallel programming, parallel-processor, parallelism, relaxation, simulation, storage interference, tasking