Bayesian optimization models for particle swarms

ABSTRACT

We explore the use of information models as a guide for the development of single objective optimization algorithms, giving particular attention to the use of Bayesian models in a PSO context. The use of an explicit information model as the basis for particle motion provides tools for designing successful algorithms. One such algorithm is developed and shown empirically to be effective. Its relationship to other popular PSO algorithms is explored and arguments are presented that those algorithms may be developed from the same model, potentially providing new tools for their analysis and tuning.

REFERENCES

• 1: M. Clerc. The swarm and the queen: Towards a deterministic and adaptive particle swarm optimization. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC 1999), pages 1951--1957, Piscataway, New Jersey, 1999.
• 2: M. Clerc. TRIBES - un exemple d'optimisation par essaim particulaire sans paramètres de contrôle. In Optimisation par Essaim Particulaire (OEP 2003), Paris, France, 2003.
• 3: M. Clerc and J. Kennedy. The particle swarm: Explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation, 6(1):58--73, February 2002.
• 4: R. C. Eberhart and Y. Shi. Comparing inertia weights and constriction factors in particle swarm optimization. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2000), pages 84--88, San Diego, California, 2000.
• 5: R. E. Kalman. A new approach to linear filtering and prediction problems. Transactions of the ASME--Journal of Basic Engineering, 82(Series D):35--45, 1960.
• 6: J. Kennedy. Bare bones particle swarms. In Proceedings of the IEEE Swarm Intelligence Symposium 2003 (SIS 2003), pages 80--87, Indianapolis, Indiana, 2003.
• 7: J. Kennedy and R. C. Eberhart. Particle swarm optimization. In International Conference on Neural Networks IV, pages 1942--1948, Piscataway, NJ, 1995. IEEE Service Center.
• 8: James Kennedy , Russell C. Eberhart, Swarm intelligence, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2001
• 9: C. T. Leondes, editor. Theory and Applications of Kalman Filtering. Number 139 in AGARDograph. North Atlantic Treaty Organization, Advisory Group for Aerospace Research and Development, 1970.
• 10: R. Mendes, J. Kennedy, and J. Neves. Watch thy neighbor or how the swarm can learn from its environment. In Proceedings of the IEEE Swarm Intelligence Symposium 2003 (SIS 2003), pages 88--94, Indianapolis, Indiana, 2003.
• 11: C. K. Monson and K. D. Seppi. The Kalman swarm. In Proceedings of the Genetic and Evolutionary Computation Conference, volume 1, pages 140--150, Seattle, Washington, 2004.
• 12: E. Ozcan and C. K. Mohan. Particle swarm optimizaton: Surfing the waves. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC 1999), Washington, D.C., 1999.
• 13: Stuart J. Russell , Peter Norvig, Artificial Intelligence: A Modern Approach, Pearson Education, 2003
• 14: Y. Shi and R. C. Eberhart. A modified particle swarm optimizer. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC 1998), Piscataway, New Jersey, 1998.
• 15: D. H. Wolpert and W. G. Macready. No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1(1):67--82, April 1997.
• 16: Y. Zheng, L. Ma, L. Zhang, and J. Qian. Empirical study of particle swarm optimizer with an increasing inertia weight. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2003), pages 221--226, Canbella, Australia, 2003.

INDEX TERMS

Primary Classification:
  G. Mathematics of Computing
  G.1 NUMERICAL ANALYSIS
      G.1.6 Optimization
          Subjects: Nonlinear programming

Additional Classification:
  G. Mathematics of Computing
  G.1 NUMERICAL ANALYSIS
      G.1.6 Optimization
          Subjects: Unconstrained optimization

General Terms:
Algorithms, Theory

Keywords:
mathematical models, optimization, swarm intelligence