Criticality dispersion in swarms to optimize n-tuples

ABSTRACT

Among numerous pattern recognition methods the neural network approach has been the subject of much research due to its ability to learn from a given collection of representative examples. This paper concerns with the optimization of a weightless neural network, which decomposes a given pattern into several sets of n points, termed n-tuples. A population-based stochastic optimization technique, known as Particle Swarm Optimization (PSO), has been used to select an optimal set of connectivity patterns to improve the recognition performance of such .n-tuple. classifiers. The original PSO was refined by combining it with a bio-inspired technique called the Self-Organized Criticality (SOC) to add diversity in the population for finding better solutions. The hybrid algorithms were adapted for the n-tuple system and the performance was measured in selecting better connectivity patterns. The aim was to improve the discriminating power of the classifier in recognizing handwritten characters by exploiting the criticality dispersion in the swarm population. This paper presents the implementation of the hybrid model in greater detail with the effect of criticality dispersion in finding better solutions.

REFERENCES

• 1: Aleksander, I., and Stonham, T.J. (1979). Guide to Pattern Recognition using Random-access Memories, Computers and Digital Techniques, vol. 2, pp. 29--40.
• 2: Azhar, M. A. H. B. and Dimond, K.R. (2004). A Stochastic Search Algorithm to Optimize an N-tuple Classifier by Selecting Its Inputs, International Conference on Image Analysis and Recognition, Porto, Portugal, Springer-Verlag, September 29 -- October 1.
• 3: Azhar, M.A.H.B., Deravi, F. and Dimond, K.R.(2005). Relative Performances of Swarm Intelligence and Genetic Algorithm to Select Better N-tuples, Proceedings of the IEEE SMC UK-RI Chapter Conference on Applied Cybernetics, pp. 111--116, London, UK, 7--8 September.
• 4: Bak, P. (1996). How nature works: the science of self-organized criticality (Copernicus, New York).
• 5: Bishop, J.M., Crowe, A.A., Minchinton, P.R. and Mitchell, R.J. (1990). Evolutionary Learning to Optimise Mapping in n-Tuple Networks, IEE Colloquium on "Machine Learning", 28 June, Digest 1990/117.
• 6: Bledsoe, W. and Browning, I. (1959). Pattern recognition and reading by machine, Proceedings of Eastern Joint Computer Conference, pp. 225--232, Birmingham.
• 7: Boettcher, S. and Paczuski, M. (1997). Aging in a Model of Self-Organized Criticality, Physical Review Letters, vol.79, Issue 5, pp. 889--892, The American Physical Society.
• 8: Deacon, J. (2006). The Really Easy Statistics Site, Biology Teaching Organisation, University of Edinburgh, http://www.biology.ed.ac.uk/research/groups/jdeacon/statistics/tress1.html
• 9: Esmin, A. A. A., Aoki, A. R., and Lambert-Torres, G. (2002). Particle swarm optimization for fuzzy membership functions optimization. Proc. of the IEEE Int. Conf. on Systems, Man and Cybernetics, pp. 108--113.
• 10: Esquivel, S.C. and Coello Coello, C.A.(2003). On the Use of Particle Swarm Optimization with Multimodal Functions. In Proceedings of the IEEE Transactions on Evolutionary Computation, vol. 2, pp. 1130--1136.
• 11: Fairhurst, M.C. and Stonham, T.J. (1976). A Class. System for Alpha-Numeric Characters Based on Learning Network Techniques, Digital Processes, vol. 2, pp. 321--339.
• 12: Garcia, L.A.C. and Souto, M. C. P. (2004). Global Optimisation Methods for Choosing the Connectivity Pattern of N-tuple Classifiers, Proc. of the IEEE International Joint Conference on Neural Networks, Budapeste, pp. 2263--2266.
• 13: Anil K. Jain , Robert P. W. Duin , Jianchang Mao, Statistical Pattern Recognition: A Review, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.1, p.4-37, January 2000 [doi>10.1109/34.824819]
• 14: Jørgensen, T. M., Christensen, S. S. and Liisberg, C. (1995). Crossvalidation and information measures for RAM based neural networks, Proc. of the Weightless Neural Networks Workshop, University of Kent at Canterbury, UK, pp. 87--92.
• 15: Thomas Martini Jørgensen , Christian Linneberg, Theoretical Analysis and Improved Decision Criteria for the n-Tuple Classifier, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.21 n.4, p.336-347, April 1999 [doi>10.1109/34.761264]
• 16: Kalyan, V., Thanmaya, P., Chilukuri, K. M. and Lisa, A. O. (2003). Optimization Using Particle Swarms with Near Neighbor Interactions, GECCO, July 11--16, Chicago, Illnois.
• 17: Kennedy, J., and Eberhart, R. C. (1995). Particle swarm optimisation, Proc. of the 1995 IEEE Int. Conf. on Neural Networks (Perth, Australia).
• 18: Krink, T. and Thomsen, R.(2001). Self--Organized Criticality and Mass Extinction in Evolutionary Algorithms, Proceedings of the Third Congress on Evolutionary Computation (CEC-2001), vol. 2, pp. 1155--1161.
• 19: M. Lovbjerg , T. Krink, Extending particle swarm optimisers with self-organized criticality, Proceedings of the Evolutionary Computation on 2002. CEC '02. Proceedings of the 2002 Congress, p.1588-1593, May 12-17, 2002
• 20: Philip Picton, Neural Networks, Macmillan Publishing Co., Inc., Indianapolis, IN, 2000
• 21: Rickers, P., Thomsen, R. and Krink, T. (2000). Applying Self-Organized Criticality to the Diffusion Model, Late Breaking Papers at the 2000 Genetic and Evolutionary Computation Conference, vol. 1, pp. 325--330, Morgan Kaufmann Publishers.
• 22: Riget, J. and Vesterstrøm, J.S. (2002). A Diversity-Guided Particle Swarm Optimizer--The ARPSO, Technical report, Department of Computer Science, University of Aarhus.
• 23: Richard Rohwer , Michal Morciniec, The theoretical and experimental status of the n-tuple classifier, Neural Networks, v.11 n.1, p.1-14, Jan. 1998
• 24: Rohwer, R. and Cressy, D. (1989). Phoneme classification by boolean networks, Proceedings of the European Conference on Speech Communication and Technology, pp. 557--560.
• 25: Settles, M. and Rylander, B. (2002).Neural network learning using particle swarm optimisers, Advances in Information Science and Soft Computing, pp. 224--226. WSEAS Press.
• 26: Yuhui Shi , Russell C. Eberhart, Parameter Selection in Particle Swarm Optimization, Proceedings of the 7th International Conference on Evolutionary Programming VII, p.591-600, March 25-27, 1998
• 27: Tukey, J.W. (1977), Exploratory Data Analysis, Reading, MA: Addison-Wesley.
• 28: Wilkinson, R., Geist, J., Janet, S., Grother, P., Burges, C., Creecy, R., Hammond, B., Hull, J., Larsen, N., Vogl, T., and Wilson, C. (1992). The first census optical character recognition systems conference, Technical Report NISTIR 4912, National Institute of Standards and Technology (NIST), Gaithersburg, USA.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.1 Models
          Subjects: Neural nets

General Terms:
Algorithms, Design, Performance

Keywords:
machine learning, optimization, pattern recognition and classification., self-organized criticality, swarm intelligence, weightless neural network