Skip to main content
Erschienen in: Neural Computing and Applications 8/2018

02.01.2017 | Original Article

A multi-verse optimizer approach for feature selection and optimizing SVM parameters based on a robust system architecture

verfasst von: Hossam Faris, Mohammad A. Hassonah, Ala’ M. Al-Zoubi, Seyedali Mirjalili, Ibrahim Aljarah

Erschienen in: Neural Computing and Applications | Ausgabe 8/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Support vector machine (SVM) is a well-regarded machine learning algorithm widely applied to classification tasks and regression problems. SVM was founded based on the statistical learning theory and structural risk minimization. Despite the high prediction rate of this technique in a wide range of real applications, the efficiency of SVM and its classification accuracy highly depends on the parameter setting as well as the subset feature selection. This work proposes a robust approach based on a recent nature-inspired metaheuristic called multi-verse optimizer (MVO) for selecting optimal features and optimizing the parameters of SVM simultaneously. In fact, the MVO algorithm is employed as a tuner to manipulate the main parameters of SVM and find the optimal set of features for this classifier. The proposed approach is implemented and tested on two different system architectures. MVO is benchmarked and compared with four classic and recent metaheuristic algorithms using ten binary and multi-class labeled datasets. Experimental results demonstrate that MVO can effectively reduce the number of features while maintaining a high prediction accuracy.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Carrizosa E, Martín-Barragán B, Morales DR (2014) A nested heuristic for parameter tuning in support vector machines. Comput Oper Res 43:328–334MathSciNetCrossRef Carrizosa E, Martín-Barragán B, Morales DR (2014) A nested heuristic for parameter tuning in support vector machines. Comput Oper Res 43:328–334MathSciNetCrossRef
2.
Zurück zum Zitat Chandrashekar G, Sahin F (2014) A survey on feature selection methods. Comput Electr Eng 40(1):16–28CrossRef Chandrashekar G, Sahin F (2014) A survey on feature selection methods. Comput Electr Eng 40(1):16–28CrossRef
3.
Zurück zum Zitat Chang C-C, Lin C-J (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):27CrossRef Chang C-C, Lin C-J (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):27CrossRef
4.
Zurück zum Zitat Chao C-F, Horng M-H (2015) The construction of support vector machine classifier using the firefly algorithm. Comput Intell Neurosci 2015:1–8CrossRef Chao C-F, Horng M-H (2015) The construction of support vector machine classifier using the firefly algorithm. Comput Intell Neurosci 2015:1–8CrossRef
5.
Zurück zum Zitat Dash M, Liu H (1997) Feature selection for classification. Intell Data Anal 1(14):131–156CrossRef Dash M, Liu H (1997) Feature selection for classification. Intell Data Anal 1(14):131–156CrossRef
6.
Zurück zum Zitat Enache A-C, Sgârciu V (2015) Anomaly intrusions detection based on support vector machines with an improved bat algorithm. In: 2015 20th international conference on control systems and computer science. pp 317–321. IEEE Enache A-C, Sgârciu V (2015) Anomaly intrusions detection based on support vector machines with an improved bat algorithm. In: 2015 20th international conference on control systems and computer science. pp 317–321. IEEE
7.
Zurück zum Zitat Faris H, Aljarah I, Mirjalili S (2016) Training feedforward neural networks using multi-verse optimizer for binary classification problems. Appl Intell 45(2):322–332CrossRef Faris H, Aljarah I, Mirjalili S (2016) Training feedforward neural networks using multi-verse optimizer for binary classification problems. Appl Intell 45(2):322–332CrossRef
8.
Zurück zum Zitat Fernández-Delgado M, Cernadas E, Barro S, Amorim D (2014) Do we need hundreds of classifiers to solve real world classification problems? J Mach Learn Res 15(1):3133–3181MathSciNetMATH Fernández-Delgado M, Cernadas E, Barro S, Amorim D (2014) Do we need hundreds of classifiers to solve real world classification problems? J Mach Learn Res 15(1):3133–3181MathSciNetMATH
9.
Zurück zum Zitat Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182MATH Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182MATH
10.
Zurück zum Zitat Han J, Kamber M, Pei J (2011) Data mining: concepts and techniques. Elsevier, AmsterdamMATH Han J, Kamber M, Pei J (2011) Data mining: concepts and techniques. Elsevier, AmsterdamMATH
11.
12.
Zurück zum Zitat Holland JH (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor Holland JH (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor
14.
Zurück zum Zitat Zhang H, Wang M, Huang X-H (2010) Parameter selection of support vector regression based on particle swarm optimization. In: 2010 IEEE international conference on granular computing, pp 834–838 Zhang H, Wang M, Huang X-H (2010) Parameter selection of support vector regression based on particle swarm optimization. In: 2010 IEEE international conference on granular computing, pp 834–838
15.
Zurück zum Zitat Huang C-L, Wang C-J (2006) A GA-based feature selection and parameters optimizationfor support vector machines. Expert Syst Appl 31(2):231–240CrossRef Huang C-L, Wang C-J (2006) A GA-based feature selection and parameters optimizationfor support vector machines. Expert Syst Appl 31(2):231–240CrossRef
16.
Zurück zum Zitat Kavousi-Fard A, Samet H, Marzbani F (2014) A new hybrid modified firefly algorithm and support vector regression model for accurate short term load forecasting. Expert Syst Appl 41(13):6047–6056CrossRef Kavousi-Fard A, Samet H, Marzbani F (2014) A new hybrid modified firefly algorithm and support vector regression model for accurate short term load forecasting. Expert Syst Appl 41(13):6047–6056CrossRef
17.
Zurück zum Zitat Kennedy J (1997) The particle swarm: social adaptation of knowledge. In: Proceedings of the 1997 international conference on evolutionary computation, pp 303–308. IEEE Service Center, Piscataway, NJ Kennedy J (1997) The particle swarm: social adaptation of knowledge. In: Proceedings of the 1997 international conference on evolutionary computation, pp 303–308. IEEE Service Center, Piscataway, NJ
18.
Zurück zum Zitat Kennedy J (1998) The behavior of particles. Evolutionary programming VII, pp 581–587 Kennedy J (1998) The behavior of particles. Evolutionary programming VII, pp 581–587
19.
Zurück zum Zitat Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Neural networks, 1995. Proceedings, IEEE international conference on, vol 4, pp 1942–1948 vol 4. IEEE Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Neural networks, 1995. Proceedings, IEEE international conference on, vol 4, pp 1942–1948 vol 4. IEEE
20.
Zurück zum Zitat Ron K, John GH (1997) Wrappers for feature subset selection. Artif Intell 97(1):273–324MATH Ron K, John GH (1997) Wrappers for feature subset selection. Artif Intell 97(1):273–324MATH
21.
Zurück zum Zitat Krishnakumar K, Goldberg DE (1992) Control system optimization using genetic algorithms. J Guid Control Dyn 15(3):735–740CrossRef Krishnakumar K, Goldberg DE (1992) Control system optimization using genetic algorithms. J Guid Control Dyn 15(3):735–740CrossRef
22.
Zurück zum Zitat Li X, Lord D, Zhang Y, Xie Y (2008) Predicting motor vehicle crashes using support vector machine models. Accid Anal Prev 40(4):1611–1618CrossRef Li X, Lord D, Zhang Y, Xie Y (2008) Predicting motor vehicle crashes using support vector machine models. Accid Anal Prev 40(4):1611–1618CrossRef
24.
Zurück zum Zitat Lin S-W, Ying K-C, Chen S-C, Lee Z-J (2008) Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst Appl 35(4):1817–1824CrossRef Lin S-W, Ying K-C, Chen S-C, Lee Z-J (2008) Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert Syst Appl 35(4):1817–1824CrossRef
25.
Zurück zum Zitat Liu H, Motoda H (1998) Feature extraction, construction and selection: a data mining perspective. Kluwer, NorwellCrossRef Liu H, Motoda H (1998) Feature extraction, construction and selection: a data mining perspective. Kluwer, NorwellCrossRef
26.
Zurück zum Zitat Liu H, Lei Y (2005) Toward integrating feature selection algorithms for classification and clustering. IEEE Trans Knowl Data Eng 17(4):491–502CrossRef Liu H, Lei Y (2005) Toward integrating feature selection algorithms for classification and clustering. IEEE Trans Knowl Data Eng 17(4):491–502CrossRef
27.
Zurück zum Zitat Maglogiannis I, Zafiropoulos E, Anagnostopoulos I (2009) An intelligent system for automated breast cancer diagnosis and prognosis using svm based classifiers. Appl Intell 30(1):24–36CrossRef Maglogiannis I, Zafiropoulos E, Anagnostopoulos I (2009) An intelligent system for automated breast cancer diagnosis and prognosis using svm based classifiers. Appl Intell 30(1):24–36CrossRef
28.
Zurück zum Zitat Mirjalili S, Mirjalili SM, Hatamlou A (2016) Multi-verse optimizer: a nature-inspired algorithm for global optimization. Neural Comput Appl 27(2):495–513CrossRef Mirjalili S, Mirjalili SM, Hatamlou A (2016) Multi-verse optimizer: a nature-inspired algorithm for global optimization. Neural Comput Appl 27(2):495–513CrossRef
29.
Zurück zum Zitat Qian Q, Gao H, Wang B (2014) A SVM method trained by improved particle swarm optimization for image classification. In: Pattern recognition. Springer, Berlin, pp 263–272 Qian Q, Gao H, Wang B (2014) A SVM method trained by improved particle swarm optimization for image classification. In: Pattern recognition. Springer, Berlin, pp 263–272
30.
Zurück zum Zitat Rodan A, Faris H (2015) Echo state network with SVM-readout for customer churn prediction. In: Applied Electrical Engineering and Computing Technologies (AEECT), 2015 IEEE Jordan Conference on, pp 1–5. IEEE Rodan A, Faris H (2015) Echo state network with SVM-readout for customer churn prediction. In: Applied Electrical Engineering and Computing Technologies (AEECT), 2015 IEEE Jordan Conference on, pp 1–5. IEEE
31.
Zurück zum Zitat Rodan A, Faris H, Alsakran J, Al-Kadi O (2014) A support vector machine approach for churn prediction in telecom industry. Inf-In Interdiscip J 17(8):3961–3970 Rodan A, Faris H, Alsakran J, Al-Kadi O (2014) A support vector machine approach for churn prediction in telecom industry. Inf-In Interdiscip J 17(8):3961–3970
33.
Zurück zum Zitat Saeys Y, Inza I, Larrañaga P (2007) A review of feature selection techniques in bioinformatics. Bioinformatics 23(19):2507–2517CrossRef Saeys Y, Inza I, Larrañaga P (2007) A review of feature selection techniques in bioinformatics. Bioinformatics 23(19):2507–2517CrossRef
34.
Zurück zum Zitat Shao Y-H, Wang Z, Chen W-J, Deng N-Y (2013) Least squares twin parametric-margin support vector machine for classification. Appl Intell 39(3):451–464CrossRef Shao Y-H, Wang Z, Chen W-J, Deng N-Y (2013) Least squares twin parametric-margin support vector machine for classification. Appl Intell 39(3):451–464CrossRef
35.
Zurück zum Zitat Sheta AF, Ahmed SEM, Faris H (2015) A comparison between regression, artificial neural networks and support vector machines for predicting stock market index. Soft Comput 7:8 Sheta AF, Ahmed SEM, Faris H (2015) A comparison between regression, artificial neural networks and support vector machines for predicting stock market index. Soft Comput 7:8
36.
Zurück zum Zitat Shin K-S, Lee TS, Kim H-J (2005) An application of support vector machines in bankruptcy prediction model. Expert Syst Appl 28(1):127–135CrossRef Shin K-S, Lee TS, Kim H-J (2005) An application of support vector machines in bankruptcy prediction model. Expert Syst Appl 28(1):127–135CrossRef
37.
Zurück zum Zitat Staelin C (2003) Parameter selection for support vector machines. Hewlett-Packard Company, Tech. Rep. HPL-2002-354R1 Staelin C (2003) Parameter selection for support vector machines. Hewlett-Packard Company, Tech. Rep. HPL-2002-354R1
38.
Zurück zum Zitat Vapnik V (1999) An overview of statistical learning theory. IEEE Trans Neural Netw 5:988–999CrossRef Vapnik V (1999) An overview of statistical learning theory. IEEE Trans Neural Netw 5:988–999CrossRef
39.
Zurück zum Zitat Vapnik V (2013) The nature of statistical learning theory. Springer, BerlinMATH Vapnik V (2013) The nature of statistical learning theory. Springer, BerlinMATH
40.
Zurück zum Zitat Wang G-G, Deb S, Cui Z (2015) Monarch butterfly optimization. Neural computing and applications, pp 1–20 Wang G-G, Deb S, Cui Z (2015) Monarch butterfly optimization. Neural computing and applications, pp 1–20
41.
Zurück zum Zitat Wang G-G, Gandomi AH, Alavi AH (2014) Stud krill herd algorithm. Neurocomputing 128:363–370CrossRef Wang G-G, Gandomi AH, Alavi AH (2014) Stud krill herd algorithm. Neurocomputing 128:363–370CrossRef
42.
Zurück zum Zitat Wang G-G, Gandomi AH, Alavi AH, Hao G-S (2014) Hybrid krill herd algorithm with differential evolution for global numerical optimization. Neural Comput Appl 25(2):297–308CrossRef Wang G-G, Gandomi AH, Alavi AH, Hao G-S (2014) Hybrid krill herd algorithm with differential evolution for global numerical optimization. Neural Comput Appl 25(2):297–308CrossRef
43.
Zurück zum Zitat Wang G-G, Gandomi AH, Zhao X, Chu HCE (2016) Hybridizing harmony search algorithm with cuckoo search for global numerical optimization. Soft Comput 20(1):273–285CrossRef Wang G-G, Gandomi AH, Zhao X, Chu HCE (2016) Hybridizing harmony search algorithm with cuckoo search for global numerical optimization. Soft Comput 20(1):273–285CrossRef
44.
Zurück zum Zitat Wang GG, Deb S, Coelho LDS (2015) Earthworm optimization algorithm: a bio-inspired metaheuristic algorithm for global optimization problems. Int J Bio-Inspired Comput (in press) Wang GG, Deb S, Coelho LDS (2015) Earthworm optimization algorithm: a bio-inspired metaheuristic algorithm for global optimization problems. Int J Bio-Inspired Comput (in press)
45.
Zurück zum Zitat Yang X-S (2010) Firefly algorithm, stochastic test functions and design optimisation. Int J Bio-Inspired Comput 2(2):78–84CrossRef Yang X-S (2010) Firefly algorithm, stochastic test functions and design optimisation. Int J Bio-Inspired Comput 2(2):78–84CrossRef
46.
Zurück zum Zitat Yang X-S (2010) A new metaheuristic bat-inspired algorithm. In: Nature inspired cooperative strategies for optimization (NICSO 2010). Springer, Berlin, pp 65–74CrossRef Yang X-S (2010) A new metaheuristic bat-inspired algorithm. In: Nature inspired cooperative strategies for optimization (NICSO 2010). Springer, Berlin, pp 65–74CrossRef
47.
Zurück zum Zitat Yang X-S, Deb S, Fong S (2011) Networked digital technologies: third international conference, NDT 2011, Macau, China, July 11–13, 2011. Proceedings, chapter Accelerated Particle Swarm Optimization and Support Vector Machine for Business Optimization and Applications. Springer, Berlin, pp 53–66CrossRef Yang X-S, Deb S, Fong S (2011) Networked digital technologies: third international conference, NDT 2011, Macau, China, July 11–13, 2011. Proceedings, chapter Accelerated Particle Swarm Optimization and Support Vector Machine for Business Optimization and Applications. Springer, Berlin, pp 53–66CrossRef
48.
Zurück zum Zitat Ye Z, Ma L, Wang M, Chen H, Zhao W (2015) Texture image classification based on support vector machine and bat algorithm. In: Intelligent data acquisition and advanced computing systems: technology and applications (IDAACS), 2015 IEEE 8th international conference on, vol 1, pp 309–314. IEEE Ye Z, Ma L, Wang M, Chen H, Zhao W (2015) Texture image classification based on support vector machine and bat algorithm. In: Intelligent data acquisition and advanced computing systems: technology and applications (IDAACS), 2015 IEEE 8th international conference on, vol 1, pp 309–314. IEEE
49.
Zurück zum Zitat Xinying Y, Liong S, Babovic V (2004) EC-SVM approach for real-time hydrologic forecasting. J Hydroinform 6:209–223CrossRef Xinying Y, Liong S, Babovic V (2004) EC-SVM approach for real-time hydrologic forecasting. J Hydroinform 6:209–223CrossRef
50.
Zurück zum Zitat Zhang XL, Chen XF, He ZJ (2010) An ACO-based algorithm for parameter optimization of support vector machines. Expert Syst Appl 37(9):6618–6628CrossRef Zhang XL, Chen XF, He ZJ (2010) An ACO-based algorithm for parameter optimization of support vector machines. Expert Syst Appl 37(9):6618–6628CrossRef
51.
Zurück zum Zitat Zhao M, Chong F, Ji L, Tang K, Zhou M (2011) Feature selection and parameter optimization for support vector machines: a new approach based on genetic algorithm with feature chromosomes. Expert Syst Appl 38(5):5197–5204CrossRef Zhao M, Chong F, Ji L, Tang K, Zhou M (2011) Feature selection and parameter optimization for support vector machines: a new approach based on genetic algorithm with feature chromosomes. Expert Syst Appl 38(5):5197–5204CrossRef
52.
Zurück zum Zitat Zhou H, Zhao JP, Zheng LG, Wang CL, Cen KF (2012) Modeling No x emissions from coal-fired utility boilers using support vector regression with ant colony optimization. Eng Appl Artif Intell 25(1):147–158CrossRef Zhou H, Zhao JP, Zheng LG, Wang CL, Cen KF (2012) Modeling No x emissions from coal-fired utility boilers using support vector regression with ant colony optimization. Eng Appl Artif Intell 25(1):147–158CrossRef
Metadaten
Titel
A multi-verse optimizer approach for feature selection and optimizing SVM parameters based on a robust system architecture
verfasst von
Hossam Faris
Mohammad A. Hassonah
Ala’ M. Al-Zoubi
Seyedali Mirjalili
Ibrahim Aljarah
Publikationsdatum
02.01.2017
Verlag
Springer London
Erschienen in
Neural Computing and Applications / Ausgabe 8/2018
Print ISSN: 0941-0643
Elektronische ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-016-2818-2

Weitere Artikel der Ausgabe 8/2018

Neural Computing and Applications 8/2018 Zur Ausgabe