GMM-PAM: a genetic multilevel multicategory perceptron associative memory

ABSTRACT

One of the principal inefficiencies in the multicategory perceptron algorithm lies in its “training algorithm”. This problem has been dealt with in the past by having multiple perceptrons trained to respond to different predefined features in the input vector using back propagation. The problem with this approach is first that in general, one cannot be sure that an appropriate set of feature vectors has been defined and second, even if it were possible to do so, one cannot insure their relative spatial geometries. A new approach for reducing the dimensionality of the pattern vector utilizes the associative memory which emerges from the cooperation among multiple distributed multicategory perceptrons connected by a genetic algorithm.

REFERENCES

• 1: Sharad Singhal , Lance Wu, Training multilayer perceptrons with the extended Kalman algorithm, Advances in neural information processing systems 1, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1989
• 2: W.S. McCulloch and W. Pitts, "A logical calculus of the ideas immanent in nervous activity,~ Bulletin of Mathematical Biophysics, vol. 5, pp. 115- 133, 1943.
• 3: J.T. Tou, and R. C. Gonzalez, Pa t tern Recognition Principles. Massachusetts: Addison-Wesley, 1974.
• 4: B. Irie and S. Miyake, "Capabilities of Threelayered Perceptrons," Proceedings of the IEEE International Conference on Neural Networks, San Diego, vol. 1, pp. 641-648, 1988.
• 5: David E. Rumelhart , James L. McClelland , CORPORATE PDP Research Group, Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations, MIT Press, Cambridge, MA, 1986
• 6: A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, and K. Lang, "Phoneme Recognition Using Time-Delay Neural Networks," ATR Internal Report TR- I-~, October 30, 1987.
• 7: D. S. Levine , P. S. Prueitt, Modeling some effects of frontal lobe damage—novelty and perseveration, Neural Networks, v.2 n.2, p.103-116, 1989 [doi>10.1016/0893-6080(89)90027-0]
• 8: J. R. Quinlan, Induction of Decision Trees, Machine Learning, v.1 n.1, p.81-106 [doi>10.1023/A:1022643204877]
• 9: Stuart Harvey Rubin, The transformative compression of coherent languages, Lehigh University, Bethlehem, PA, 1988
• 10: S.H. Rubin, On the Non-Computability of an Auto-Transformative Mechanics. Unpublished manuscript, Central Michigan University, Department of Computer Science, Mt. Pleasant, MI. 1988b.
• 11: D.H. Hubel and T. N. Wiesel, Receptive Fields, "Binocular Interaction and Functional Architecture in the Cat's Visual Cortex," Journal of Physiology, vol. 160 pp. 106-154, 1962.
• 12: T. Kohonen, "Self-organized formation of topologically correct feature maps," Biological Cybernetics, vol. 43, pp. 59-69, 1982.
• 13: T. Kohonen, "Adaptive, Associative and Selforganizing Functions in Neural Computing," Applied Optics, vol. 26, pp. 4910-4918, 1987.
• 14: John H. Holland, Genetic algorithms and classifier systems: foundations and future directions, Proceedings of the Second International Conference on Genetic Algorithms on Genetic algorithms and their application, p.82-89, October 1987, Cambridge, Massachusetts, United States
• 15: J.C. Eccles, The Understanding of the Brain. New York: McGraw-Hill, 1977.
• 16: M. Bichsel , P. Seitz, Minimum class entropy: a maximum information approach to layered networks, Neural Networks, v.2 n.2, p.133-141, 1989 [doi>10.1016/0893-6080(89)90030-0]
• 17: J. S. Denker , W. R. Gardner , H. P. Graf , D. Henderson , R. E. Howard , W. Hubbard , L. D. Jackel , H. S. Baird , I. Guyon, Neural network recognizer for hand-written zip code digits, Advances in neural information processing systems 1, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1989
• 18: K. Fukushima, S. Miyake, and T. Ito, "Neocognitron: A Neural Network Model for a Mechanism of Visual Pattern Recognition," IEEE Transactions on Systems, Man, and Cybernetics, vol. 13, no. 5, pp. 826-834, 1983.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE

Additional Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Probabilistic algorithms (including Monte Carlo)

  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Pattern analysis

General Terms:
Algorithms