Relating number of processing elements in a sparse distributed memory model to learning rate and generalization

ABSTRACT

A simulated neural network was developed with APL on an 80386 microcomputer. The network was configured to associate task descriptions with 10 categories of military occupational specialties. The number of processing elements in the problem was varied. Increasing the number of processors increased the speed of learning in the simulation. Generalization was not significantly different for various numbers of processing elements except for one intermediate number at which generalization occurred about 15 percent higher. Analysis of the performance of a trained network suggests that low level, natural language understanding is one form of text processing which promises to become an important application area for neural model-based computing.

REFERENCES

• 1: C. Bereiter, "Implications of Connectionism for Thinking about Rules," Educational Researcher, vol. 20, no. 3, pp. 10-16, (April, 1991.)
• 2: L. Cartwright, "Task classification: Proposed Approach and Applications," Paper presented to the Defense Training and Performance Data Center, Orlando, Florida, (September, 1990.)
• 3: G. DeJong, " ' Skimming Stories in Real Time: An Experiment in Integrated Understanding," Research Report 158, Department of Computer Science, Yale University, (1979).
• 4: E. R. Hilgard and G. H. Bower, Theories of Learning, Appelton-Century-Crofts, New York, (1956).
• 5: P. S. Jacobs , Lisa F. Rau, SCISOR: extracting information from on-line news, Communications of the ACM, v.33 n.11, p.88-97, Nov. 1990 [doi>10.1145/92755.92769]
• 6: Pentti Kanerva, Sparse Distributed Memory, MIT Press, Cambridge, MA, 1988
• 7: J. Martin, "Text Management's Mission: Locate What's Relevant," PC Week, p. 68, (27 August 1990.)
• 8: Edward Rietman, Experiments in Artificial Neural Networks, McGraw-Hill, Inc., New York, NY, 1988
• 9: B. Sudheim, "Second Message Understanding Conference (MUCK-II) Test Report," Technical Report 1328, Naval Ocean Systems Center, San Diego, CA, 1990.
• 10: A. Surkan, APL descriptions of functional building blocks for connectionist computer models, ACM SIGAPL APL Quote Quad, v.19 n.4, p.334-340, Aug. 1989
• 11: S. Young and P. Hayes, "Automatic Classification and Summarization of Banking Telex," The Second Conference on Artificial Intelligence Applications. Washington, DC: IEEE Computer Society Press, pp. 402-408, (1985).

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING
      I.6.8 Types of Simulation
          Subjects: Distributed

Additional Classification:
  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.2 Language Classifications

          Nouns: APL

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
          Subjects: Connectionism and neural nets
      I.2.7 Natural Language Processing
          Subjects: Text analysis

General Terms:
Design, Languages, Performance