The impact of network topology on self-organizing maps

ABSTRACT

In this paper, we study instances of complex neural networks, i.e. neural networks with complex topologies. We use Self-Organizing Map neural networks whose neighborhood relationships are defined by a complex network, to classify handwritten digits. We show that topology has a small impact on performance and robustness to neuron failures, at least at long learning times. Performance may however be increased (by almost $10\%$) by evolutionary optimization of the network topology. In our experimental conditions, the evolved networks are more random than their parents, but display a more heterogeneous degree distribution.

REFERENCES

• 1: {1} A.-L. Barabasi and R. Albert. Emergence of scaling in random networks. Science, 286:509, 1999.
• 2: S. Boccaletti, V. Latora, Y. Moreno, M. Chavez, and D.U. Hwang. Complex networks: Structure and dynamics. Physics Reports, 424:175--308, 2006.
• 3: J. Bohland and A. Minai. Efficient associative memory using small-world architecture. Neurocomputing, 38--40:489--496, 2001.
• 4: Z. Deng and Y. Zhang. Complex systems modeling using scale-free highly-clustered echo state network. In IJCNN'06, pages 3128--3135, 2006.
• 5: Agoston E. Eiben , J. E. Smith, Introduction to Evolutionary Computing, SpringerVerlag, 2003
• 6: B.J. Kim. Performance of networks of artificial neurons: the role of clustering. Phys. Rev. E, 64:045101, 2004.
• 7: J. M. Kleinberg. Navigation in a small world. Nature, 406(6798), 2000.
• 8: T. Kohonen, Self-organization and associative memory: 3rd edition, Springer-Verlag New York, Inc., New York, NY, 1989
• 9: Yann LeCun and Corinna Cortes. The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist/.
• 10: C. Darabos M. Tomassini, M. Giacobini. Evolution and dynamics of small--world cellular automata. Complex Systems, 15:261--284, 2005.
• 11: P.N. McGraw and M. Menzinger. Topology and computational performance of attractor neural networks. Phys. Rev. E, 68:047102, 2003.
• 12: L. G. Morelli, G. Abramson, and M. N. Kuperman. Associative memory on a small-world neural network. Eur. Phys. J. B, 38:495--500, 2004.
• 13: S. Nolfi and D. Parisi. Handbook of brain theory and neural networks, chapter Evolution of artificial neural networks, pages 418--421. MIT Press, 2002.
• 14: P. Oikonomou and P. Cluzel. Eýects of topology on network evolution. Nature Physics, 2:532--536, 2006.
• 15: D Simard, L Nadeau, and H. Kroger. Fastest learning in small-world neural networks. Phys. Lett. A, 336:8--15, 2005.
• 16: Kenneth O. Stanley , Risto Miikkulainen, Evolving neural networks through augmenting topologies, Evolutionary Computation, v.10 n.2, p.99-127, Summer 2002 [doi>10.1162/106365602320169811]
• 17: D. Stauýer, A. Aharony, L da Fontoura Costa, and J Adler. Efficient Hopfield pattern recognition on a scale-free NN. Eur. Phys. J. B, 32:395--399, 2003.
• 18: S. H. Strogatz. Exploring complex networks. Nature, 410(6825):268--276, 2001.
• 19: Thomas Villmann , Erzsébet Merényi, Extensions and modifications of the Kohenen-SOM and applications in remote sensing image analysis, Self-Organizing neural networks: recent advances and applications, Springer-Verlag New York, Inc., New York, NY, 2001
• 20: D.J. Watts and S. H. Strogatz. Collective dynamics of 'small-world' networks. Nature, 393:440--442, 1998.
• 21: X. Yao. Evolving artificial neural networks. Proc. IEEE, 87:1423--1447, 1999.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
          Subjects: Connectionism and neural nets

Additional Classification:
  G. Mathematics of Computing
  G.1 NUMERICAL ANALYSIS
      G.1.6 Optimization
          Subjects: Stochastic programming

General Terms:
Algorithms, Design, Experimentation

Keywords:
evolutionary optimization, self-organizing map, topology