Identification of parallelism in neural networks by simulation with language J.

ABSTRACT

Neural networks, trained by backpropagation, are designed and described in the language J, an APL derivative with powerful function encapsulation features. Both the languages J [4,6,7] and APL [5] help to identify and isolate the parallelism that is inherent in network training algorithms. Non-critical details of data input and derived output processes are de-emphasized by relegating those functions to callable stand-alone modules. Such input and output modules can be isolated and customized individually for managing communication with arbitrary, external storage systems. The central objective of this research is the design and precise description of a neural network training kernel. Such kernel designs are valuable for producing efficient reusable computer codes and facilitating the transfer of neural network technology from developers to users.

REFERENCES

• 1: Robert Bernecky, Function arrays, Proceedings of the international conference on APL, p.53-56, June 11-15, 1984, Finland
• 2: Robert Bernecky , Roger K. W. Hui, Gerunds and representations, ACM SIGAPL APL Quote Quad, v.21 n.4, p.39-46, Aug. 1991
• 3: Frey, R.J. "Object Oriented Extensions to APL and J", Vector, Vol.9, No.2 1992
• 4: Roger K. W. Hui , Kenneth E. Iverson , E. E. McDonnell , Arthur T. Whitney, APL\?, ACM SIGAPL APL Quote Quad, v.20 n.4, p.192-200, July 1990
• 5: K. E. Iverson, A personal view of APL, IBM Systems Journal, v.30 n.4, p.582-593, 1991
• 6: Iverson, K.E. "Programming in J" ISI 1991
• 7: D. B. McIntyre, Language as an intellectual tool: from hieroglyphics to APL, IBM Systems Journal, v.30 n.4, p.554-581, 1991
• 8: David E. Rumelhart , James L. McClelland , CORPORATE PDP Research Group, Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations, MIT Press, Cambridge, MA, 1986
• 9: Alexei N. Skurihin, Neural networks in J, ACM SIGAPL APL Quote Quad, v.23 n.1, p.216-220, July 1992
• 10: Werbos, P.J. "Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences", Ph.D. Thesis, Harvard University, Cambridge, MA. August (1974)
• 11: Yong Yao , Walter J. Freeman , Brian Burke , Qing Yang, Pattern recognition by a distributed neural network: an industrial application, Neural Networks, v.4 n.1, p.103-121, 1991 [doi>10.1016/0893-6080(91)90036-5]

INDEX TERMS

Primary Classification:
  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.2 Language Classifications

          Nouns: APL

Additional Classification:
  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.3 Concurrent Programming
          Subjects: Parallel programming

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
          Subjects: Connectionism and neural nets
  I.6 SIMULATION AND MODELING
      I.6.8 Types of Simulation
          Subjects: Parallel

General Terms:
Algorithms, Languages