WSDM: Weighted sparse distributed memory prototype expressed in APL

ABSTRACT

A functional style application of APL notation succinctly describes the architecture and principles of operation of one kind of connection-based computer. In the future it is expected that these machines will have thousands of processors and large arrays of dynamic connections. APL programs running on von Neumann computers now provide precise descriptions of connection-based machines which are convenient for exploring the potential of connection-based computation. Experience with radically new structures and different principles of operation for neural network problem solving can be obtained using virtual machines provided by software. Virtual machines are described by functions programmed on conventional computers. Two adaptive variants of the sparse distributed memory or SDM (Kanerva [1991]) show improved efficiency. The demonstrated superiority of Kanerva's new pattern weighting idea can be obtained by improved coding of the input patterns. This coding is done by generally defined preprocessing of features of representative binary input patterns. Transformed input patterns select addresses which pack distributed memories more efficiently. Coding is done by first computing customized weight vectors for each input pattern vector. Individual weighting of each pattern leads to more uniform utilization of the addresses and their corresponding memory connection weights. The derived pattern weights improve discrimination between pairs of similar inputs with few significant differences. This paper is to provide the APL community access to a concise symbolic description of Kanerva's weighted SDM machine. APL's rich set of computer modeling and exposition tools have the potential of markedly accelerating software and hardware development for array-base connectionist computing.

REFERENCES

• 1: Pentti Kanerva, Sparse Distributed Memory, MIT Press, Cambridge, MA, 1988
• 2: Kanerva, P.,"Efficient Packing of Patterns in Sparse Distributed Memory by Selective Weighting of Input Bits" Artificial Neural Networks 1991, Volume I, pp. 279-284, Proceedings of ICANN-91, International Conference on Artificial Neural Networks - Espoo, Finland, June 24-28, 1991, editors: T. Kohonen, K. Makisara, O. Simula and J. Kangus. (Amsterdam: North Holland, 1991).
• 3: Danforth, D., An Empirical Investigation of Sparse Distributed Memory Using Discrete Speech Recognition, in: Proceedings of International Neural -Network Conference, Pads, France, July 9-13, 1 (1990), pp. 183-186 (Kluwer Academic Publishers, Dordrecht, The Netherlands, 1990)
• 4: Keeler, J., Capacity for Patterns and Sequences in Kanerva's SDM as Compared to Other Associative Memory Models, in: Anderson, D., (ed), Neural Information Processing Models (American Institute of Physics Press, New York, 1988)
• 5: Prager, R. W., Clarke, T. J. W. and FaUside, F., The Modified Kanerva Model: Results for Real Time Word Recognition, in: IEEE First International Conference on Artificial Neural Networks, Savoy Place, London, 16-18 October 1989
• 6: Sa~nen, J., Pohja, S., and Kaski, K., "Self- Organization with Kanerva's Sparse Distributed Memory" Artificial Neural Networks 1991, Volume I, pp. 284-289, ProceeAings of ICANN-91, International Conference on Artificial Neural Networks - Espoo, Finland, June 24-28, 1991, editors: T. Kohonen, K. Makisara, O. Simula and J. Kangus. (Amsterdam: North Holland, 1991).
• 7: Surkan "Application of Neural Networks to Classification of Binary Prof'fles Derived from Individual Interviews" pp. 467-472 IEEE International Conference on Neural Networks July 24-27, 1988.
• 8: Surkan "Fast Trainable Pattern Classification by a Modification of Kanerva's SDM" (with L. Di) In~ern~ti0nal Joint Conference on Neural Networks, Volume 1, July 18, 1989, pp. 342-350; IEEE TAB Neural Network Committee, San Diego.

INDEX TERMS

Primary Classification:
  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.2 Language Classifications

          Nouns: APL

Additional Classification:
  C. Computer Systems Organization
  C.1 PROCESSOR ARCHITECTURES
      C.1.2 Multiple Data Stream Architectures (Multiprocessors)
          Subjects: Connection machines

General Terms:
Experimentation, Languages