Skip to main content
Erschienen in: Journal of Intelligent Information Systems 3/2020

05.09.2019

A novel multi-strategy DE algorithm for parameter optimization in support vector machine

verfasst von: Kangjing Li, Jiaxiang Luo, Yueming Hu, Shipeng Li

Erschienen in: Journal of Intelligent Information Systems | Ausgabe 3/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Support vector machine (SVM) is a powerful technique in pattern classification, but its performance is highly dependent on its parameters. In this paper, a new SVM optimized by a novel differential evolution (DE) with a hybrid parameter setting strategy and a population size adaptation method is proposed and simplified as FDE-PS-SVM. In the hybrid parameter setting strategy, the SVM parameter offspring are generated by DE operators with evolutionary parameters that are fixed or with the ones generated by fuzzy logic inference (FLI) according to a given probability. In the population size adaptation method, the population size is shrunk gradually during the search, which tries to balance the diversity and concentration ability of the algorithm to find better SVM parameters. Some benchmark data sets are used to evaluate the proposed algorithm. Experimental results show that the two proposed strategies are effective to search for better SVM parameters while the proposed FDE-PS-SVM algorithm outperforms other algorithms published in other literature.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Adankon, M.M., & Cheriet, M. (2009). Model selection for the LS-SVM. Application to handwriting recognition. Pattern Recognition, 42(12), 3264–3270.MATHCrossRef Adankon, M.M., & Cheriet, M. (2009). Model selection for the LS-SVM. Application to handwriting recognition. Pattern Recognition, 42(12), 3264–3270.MATHCrossRef
Zurück zum Zitat Ahmad, A.S., Hassan, M.Y., Abdullah, M.P., Rahman, H.A., Hussin, F., Abdullah, H., Saidur, R. (2014). A review on applications of ANN and SVM for building electrical energy consumption forecasting. Renewable and Sustainable Energy Reviews, 33, 102–109.CrossRef Ahmad, A.S., Hassan, M.Y., Abdullah, M.P., Rahman, H.A., Hussin, F., Abdullah, H., Saidur, R. (2014). A review on applications of ANN and SVM for building electrical energy consumption forecasting. Renewable and Sustainable Energy Reviews, 33, 102–109.CrossRef
Zurück zum Zitat Alon, U., Barkai, N., Notterman, D.A., Gish, K., Ybarra, S., Mack, D., et al. (1999). Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proceedings of the National Academy of Sciences of the United States of America, 96(12), 6745–6750.CrossRef Alon, U., Barkai, N., Notterman, D.A., Gish, K., Ybarra, S., Mack, D., et al. (1999). Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proceedings of the National Academy of Sciences of the United States of America, 96(12), 6745–6750.CrossRef
Zurück zum Zitat Aydin, I., Karakose, M., Akin, E. (2011). A multi-objective artificial immune algorithm for parameter optimization in support vector machine. Applied Soft Computing, 11(1), 120–129.CrossRef Aydin, I., Karakose, M., Akin, E. (2011). A multi-objective artificial immune algorithm for parameter optimization in support vector machine. Applied Soft Computing, 11(1), 120–129.CrossRef
Zurück zum Zitat Brest, J., Greiner, S., Boskovic, B., Mernik, M., Zumer, V. (2006). Self-adapting control parameters in differential evolution: A comparative study on numerical benchmark problems. IEEE Transactions on Evolutionary Computation, 10(6), 646–657.CrossRef Brest, J., Greiner, S., Boskovic, B., Mernik, M., Zumer, V. (2006). Self-adapting control parameters in differential evolution: A comparative study on numerical benchmark problems. IEEE Transactions on Evolutionary Computation, 10(6), 646–657.CrossRef
Zurück zum Zitat Burges, C.J.C. (1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2(2), 121–167.CrossRef Burges, C.J.C. (1998). A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2(2), 121–167.CrossRef
Zurück zum Zitat Chang, C.C., & Lin, C.J. (2011). Libsvm: a library for support vector machines. Acm Transactions on Intelligent Systems & Technology, 2(3), 27:1 – 27:27.CrossRef Chang, C.C., & Lin, C.J. (2011). Libsvm: a library for support vector machines. Acm Transactions on Intelligent Systems & Technology, 2(3), 27:1 – 27:27.CrossRef
Zurück zum Zitat Chapelle, O., Vapnik, V., Bousquet, O., Mukherjee, S. (2002). Choosing multiple parameters for support vector machines. Machine Learning, 46(1-3), 131–159.MATHCrossRef Chapelle, O., Vapnik, V., Bousquet, O., Mukherjee, S. (2002). Choosing multiple parameters for support vector machines. Machine Learning, 46(1-3), 131–159.MATHCrossRef
Zurück zum Zitat Das, S., & Suganthan, P.N. (2011). Differential evolution: a survey of the state-of-the-art. IEEE Transactions on Evolutionary Computation, 15(1), 4–31.CrossRef Das, S., & Suganthan, P.N. (2011). Differential evolution: a survey of the state-of-the-art. IEEE Transactions on Evolutionary Computation, 15(1), 4–31.CrossRef
Zurück zum Zitat Duan, K., Keerthi, S.S., Poo, A.N. (2003). Evaluation of simple performance measures for tuning svm hyperparameters. Pattern Recognition, 51, 41–59. Duan, K., Keerthi, S.S., Poo, A.N. (2003). Evaluation of simple performance measures for tuning svm hyperparameters. Pattern Recognition, 51, 41–59.
Zurück zum Zitat Eitrich, T., & Lang, B. (2006). Efficient optimization of support vector machine learning parameters for unbalanced datasets. Journal of Computational & Applied Mathematics, 196(2), 425–436.MathSciNetMATHCrossRef Eitrich, T., & Lang, B. (2006). Efficient optimization of support vector machine learning parameters for unbalanced datasets. Journal of Computational & Applied Mathematics, 196(2), 425–436.MathSciNetMATHCrossRef
Zurück zum Zitat Guyon, I., Weston, J., Barnhill, S., Vapnik, V. (2002). Gene selection for cancer classification using support vector machines. Machine Learning, 46(1-3), 389–422.MATHCrossRef Guyon, I., Weston, J., Barnhill, S., Vapnik, V. (2002). Gene selection for cancer classification using support vector machines. Machine Learning, 46(1-3), 389–422.MATHCrossRef
Zurück zum Zitat Hsu, C.W., & Lin, C.J. (2002). A comparison of methods for multi-class support vector machines. IEEE Transactions on Neural Networks, 13(4), 41–425. Hsu, C.W., & Lin, C.J. (2002). A comparison of methods for multi-class support vector machines. IEEE Transactions on Neural Networks, 13(4), 41–425.
Zurück zum Zitat Huang, C.L. (2009). Aco-based hybrid classification system with feature subset selection and model parameters optimization. Neurocomputing, 73(1-3), 438–448.CrossRef Huang, C.L. (2009). Aco-based hybrid classification system with feature subset selection and model parameters optimization. Neurocomputing, 73(1-3), 438–448.CrossRef
Zurück zum Zitat Huang, C.L., & Wang, C.J. (2006). A ga-based feature selection and parameters optimization for support vector machines. Expert Systems with Applications, 31(2), 231–240.CrossRef Huang, C.L., & Wang, C.J. (2006). A ga-based feature selection and parameters optimization for support vector machines. Expert Systems with Applications, 31(2), 231–240.CrossRef
Zurück zum Zitat Keerthi, S.S., & Lin, C.J. (2003). Asymptotic behaviors of support vector machines with gaussian kernel. Neural Computation, 15(7), 1667–1689.MATHCrossRef Keerthi, S.S., & Lin, C.J. (2003). Asymptotic behaviors of support vector machines with gaussian kernel. Neural Computation, 15(7), 1667–1689.MATHCrossRef
Zurück zum Zitat Keerthi, S.S., Sindhwani, V., Chapelle, O. (2007). An efficient method for gradient-based adaptation of hyperparameters in svm models. Advances in Neural Information Processing Systems 19:Proceedings of the 2006 Conference, 42(12), 673–680. Keerthi, S.S., Sindhwani, V., Chapelle, O. (2007). An efficient method for gradient-based adaptation of hyperparameters in svm models. Advances in Neural Information Processing Systems 19:Proceedings of the 2006 Conference, 42(12), 673–680.
Zurück zum Zitat Khan, N.M., Ksantini, R., Ahmad, I.S., Boufama, B. (2012). A novel svm+nda model for classification with an application to face recognition. Pattern Recognition, 45 (1), 66–79.MATHCrossRef Khan, N.M., Ksantini, R., Ahmad, I.S., Boufama, B. (2012). A novel svm+nda model for classification with an application to face recognition. Pattern Recognition, 45 (1), 66–79.MATHCrossRef
Zurück zum Zitat Li, S.T., Kwok, J.T., Zhu, H.L. (2003). Texture classification using the support vector machines. Pattern Recognition, 36(12), 2883–2893.MATHCrossRef Li, S.T., Kwok, J.T., Zhu, H.L. (2003). Texture classification using the support vector machines. Pattern Recognition, 36(12), 2883–2893.MATHCrossRef
Zurück zum Zitat Li, J., Ding, L.X., Xing, Y. (2013). Differential evolution based parameters selection for support vector machine. In: International Conference on Computational Intelligence & Security, pp. 284–288. Li, J., Ding, L.X., Xing, Y. (2013). Differential evolution based parameters selection for support vector machine. In: International Conference on Computational Intelligence & Security, pp. 284–288.
Zurück zum Zitat Lin, S.W., Ying, K.C., Chen, S.C., Lee, Z.J. (2008). Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Systems with Applications, 35(4), 1817–1824.CrossRef Lin, S.W., Ying, K.C., Chen, S.C., Lee, Z.J. (2008). Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Systems with Applications, 35(4), 1817–1824.CrossRef
Zurück zum Zitat Liu, J.H., & Lampinen, J. (2003). Population size adaptation for differential evolution algorithm using fuzzy logic. Intelligent Systems Design and Applications, 42 (12), 3264–3270. Liu, J.H., & Lampinen, J. (2003). Population size adaptation for differential evolution algorithm using fuzzy logic. Intelligent Systems Design and Applications, 42 (12), 3264–3270.
Zurück zum Zitat Liu, J., & Lampinen, J. (2005). A fuzzy adaptive differential evolution algorithm. Soft Computing, 9(6), 448–462.MATHCrossRef Liu, J., & Lampinen, J. (2005). A fuzzy adaptive differential evolution algorithm. Soft Computing, 9(6), 448–462.MATHCrossRef
Zurück zum Zitat Liu, R.J., Wang, Y.H., Baba, T., Masumoto, D., Nagata, S. (2008). Svm-based active feedback in image retrieval using clustering and unlabeled data. Pattern Recognition, 41(8), 2645–2655.MATHCrossRef Liu, R.J., Wang, Y.H., Baba, T., Masumoto, D., Nagata, S. (2008). Svm-based active feedback in image retrieval using clustering and unlabeled data. Pattern Recognition, 41(8), 2645–2655.MATHCrossRef
Zurück zum Zitat Olatomiwa, L., Mekhilef, S., Shamshirband, S., Mohammadi, K., Petković, D., Sudheer, C. (2015). A support vector machine–firefly algorithm-based model for global solar radiation prediction. Solar Energy, 115, 632–644.CrossRef Olatomiwa, L., Mekhilef, S., Shamshirband, S., Mohammadi, K., Petković, D., Sudheer, C. (2015). A support vector machine–firefly algorithm-based model for global solar radiation prediction. Solar Energy, 115, 632–644.CrossRef
Zurück zum Zitat Qin, A.K., & Suganthan, P.N. (2005). Self-adaptive differential evolution algorithm for numerical optimization. 2005 IEEE Congress on Evolutionary Computation, 2, 1785–1791.CrossRef Qin, A.K., & Suganthan, P.N. (2005). Self-adaptive differential evolution algorithm for numerical optimization. 2005 IEEE Congress on Evolutionary Computation, 2, 1785–1791.CrossRef
Zurück zum Zitat Qin, A.K., Huang, V.L., Suganthan, P.N. (2009). Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Transactions on Evolutionary Computation, 13(2), 398–417.CrossRef Qin, A.K., Huang, V.L., Suganthan, P.N. (2009). Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Transactions on Evolutionary Computation, 13(2), 398–417.CrossRef
Zurück zum Zitat Santos, G.S.D., Luvizotto, L.G.J., Mariani, V.C., Coelho, L.D.S. (2012). Least squares support vector machines with tuning based on chaotic differential evolution approach applied to the identification of a thermal process. Expert Systems with Applications, 39(5), 4805–4812.CrossRef Santos, G.S.D., Luvizotto, L.G.J., Mariani, V.C., Coelho, L.D.S. (2012). Least squares support vector machines with tuning based on chaotic differential evolution approach applied to the identification of a thermal process. Expert Systems with Applications, 39(5), 4805–4812.CrossRef
Zurück zum Zitat Sarker, R.A., Elsayed, S.M., Ray, T. (2014). Differential evolution with dynamic parameters selection for optimization problems. IEEE Transactions on Evolutionary Computation. Sarker, R.A., Elsayed, S.M., Ray, T. (2014). Differential evolution with dynamic parameters selection for optimization problems. IEEE Transactions on Evolutionary Computation.
Zurück zum Zitat Scholkopf, B., Guyon, I., Weston, J. (2001). Statistical learning and kernel methods in bioinformatics. International Centre for Mechanical Sciences, 6(97), 111–120. Scholkopf, B., Guyon, I., Weston, J. (2001). Statistical learning and kernel methods in bioinformatics. International Centre for Mechanical Sciences, 6(97), 111–120.
Zurück zum Zitat Storn, R., & Price, K. (1997). Differential evolution - a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization, 11(4), 341–359.MathSciNetMATHCrossRef Storn, R., & Price, K. (1997). Differential evolution - a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization, 11(4), 341–359.MathSciNetMATHCrossRef
Zurück zum Zitat Wolpert, D.H., & Macready, W.G. (1997). No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1(1), 67–82.CrossRef Wolpert, D.H., & Macready, W.G. (1997). No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1(1), 67–82.CrossRef
Zurück zum Zitat Wu, C.H., Tzeng, G.H., Goo, Y.J., Fang, W.C. (2007). A real-valued genetic algorithm to optimize the parameters of support vector machine for predicting bankruptcy. Wu, C.H., Tzeng, G.H., Goo, Y.J., Fang, W.C. (2007). A real-valued genetic algorithm to optimize the parameters of support vector machine for predicting bankruptcy.
Zurück zum Zitat Xia, S.X., Lin, R., Cui, X., Shan, J. (2016). The application of orthogonal test method in the parameters optimization of fPEMFCg under steady working condition. International Journal of Hydrogen Energy, 41(26), 11380–11390.CrossRef Xia, S.X., Lin, R., Cui, X., Shan, J. (2016). The application of orthogonal test method in the parameters optimization of fPEMFCg under steady working condition. International Journal of Hydrogen Energy, 41(26), 11380–11390.CrossRef
Zurück zum Zitat Yuana, S., & Chua, F. (2007). Fault diagnosis based on support vector machines with parameter optimisation by artificial immunisation algorithm. Mechanical Systems and Signal Processing, 21(3), 1318–1330.CrossRef Yuana, S., & Chua, F. (2007). Fault diagnosis based on support vector machines with parameter optimisation by artificial immunisation algorithm. Mechanical Systems and Signal Processing, 21(3), 1318–1330.CrossRef
Zurück zum Zitat Zhang, J., & Sanderson, A.C. (2009). Jade: Adaptive differential evolution with optional external archive. IEEE Transactions on Evolutionary Computation, 13(5), 945–958.CrossRef Zhang, J., & Sanderson, A.C. (2009). Jade: Adaptive differential evolution with optional external archive. IEEE Transactions on Evolutionary Computation, 13(5), 945–958.CrossRef
Zurück zum Zitat Zhang, X.Y., Zhou, J.Z., Wang, C.Q., Li, C.S., Song, L.X. (2012). Multi-class support vector machine optimized by inter-cluster distance and self-adaptive deferential evolution. Applied Mathematics and Computation, 218(9), 4973–4987.MathSciNetMATHCrossRef Zhang, X.Y., Zhou, J.Z., Wang, C.Q., Li, C.S., Song, L.X. (2012). Multi-class support vector machine optimized by inter-cluster distance and self-adaptive deferential evolution. Applied Mathematics and Computation, 218(9), 4973–4987.MathSciNetMATHCrossRef
Zurück zum Zitat Zhang, X.Y., Qiu, D.Y., Chen, F.A. (2015). Support vector machine with parameter optimization by a novel hybrid method and its application to fault diagnosis. Neurocomputing, 149, 641–651.CrossRef Zhang, X.Y., Qiu, D.Y., Chen, F.A. (2015). Support vector machine with parameter optimization by a novel hybrid method and its application to fault diagnosis. Neurocomputing, 149, 641–651.CrossRef
Metadaten
Titel
A novel multi-strategy DE algorithm for parameter optimization in support vector machine
verfasst von
Kangjing Li
Jiaxiang Luo
Yueming Hu
Shipeng Li
Publikationsdatum
05.09.2019
Verlag
Springer US
Erschienen in
Journal of Intelligent Information Systems / Ausgabe 3/2020
Print ISSN: 0925-9902
Elektronische ISSN: 1573-7675
DOI
https://doi.org/10.1007/s10844-019-00573-w

Weitere Artikel der Ausgabe 3/2020

Journal of Intelligent Information Systems 3/2020 Zur Ausgabe