Parameters optimization of support vector regression based on immune particle swarm optimization algorithm

ABSTRACT

A novel Immune Particle Swarm Optimization (IPSO) for parameters optimization of Support Vector Regression (SVR) is proposed in this article. After introduced clonal copy and mutation process of Immune Algorithm (IA), the particle of PSO is considered as antibodies. Therefore, evaluated the fitness of particles by the Leave-One-Out Cross-Validation (LOOCV) standard, the best individual mutated particle for each cloned group will be selected to compose the next generation to get better parameters of SVR. It can construct high accuracy and generalization performance regression model rapidly by optimizing the combination of three SVR parameters at the same time. Under the datasets generated from sinx function with additive noise and spectra dataset, simulation results show that the new method can determine the parameters of SVR quickly and the gotten models have superior learning accuracy and generalization performance.

REFERENCES

• 1: Vladimir N. Vapnik, The nature of statistical learning theory, Springer-Verlag New York, Inc., New York, NY, 1995
• 2: Vapnik, V. 1999, An overview of Statistical Learning Theory. IEEE Trans. On Neural Networks, 10(5):988--999,
• 3: Kristin P. Bennett , Colin Campbell, Support vector machines: hype or hallelujah?, ACM SIGKDD Explorations Newsletter, v.2 n.2, p.1-13, Dec. 2000 [doi>10.1145/380995.380999]
• 4: Alex J. Smola , Bernhard Schölkopf, A tutorial on support vector regression, Statistics and Computing, v.14 n.3, p.199-222, August 2004 [doi>10.1023/B:STCO.0000035301.49549.88]
• 5: Kennedy J. and Eberhart, R. C. 1995, "Particle Swarm Optimization," in Proc. IEEE Conf. Neural Networks IV, Vol. 4, pp. 1942--1948.
• 6: Shi, Y., and Eberhart, R. C., 1998, "A Modified Particle Swarm Optimizer," Proceedings of the IEEE International Conference on Evolutionary Computation, pp. 69--73.
• 7: R.C. Eberhart, and Shi, Y. 2001, Particle Swarm Optimization: Developments, Applications and Resources, in: Proceedings of International Conference on Evolutionary Computation, pp. 81--86.
• 8: John E. Hunt , Denise E. Cooke, Learning using an artificial immune system, Journal of Network and Computer Applications, v.19 n.2, p.189-212, April 1996 [doi>10.1006/jnca.1996.0014]
• 9: Vladimir Cherkassky , Yunqian Ma, Practical selection of SVM parameters and noise estimation for SVM regression, Neural Networks, v.17 n.1, p.113-126, January 2004 [doi>10.1016/S0893-6080(03)00169-2]
• 10: Xin Wang, Chunhua Yang, Bin Qin, Weihua Gui. 2005, Parameter selection of support vector regression based on hybrid optimization algorithm and its application. Journal of Control Theory and Applications, 2005, 4 : 371--376
• 11: Kalivas, John H., 1997, "Two Data Sets of Near Infrared Spectra," Chemometrics and Intelligent Laboratory Systems, v.37 (1997) pp.255--259
• 12: Haenlein, Michael and Andreas M. Kaplan , 2004, "A Beginner's Guide to Partial Least Squares Analysis, Understanding Statistics, 3(4), 283--297".

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
          Subjects: Parameter learning

General Terms:
Algorithms, Experimentation

Keywords:
immune algorithm, parameters optimization, particle swarm optimization, support vector regression