Skip to main content
Erschienen in: Neural Processing Letters 1/2016

01.08.2016

Hybrid Harmony Search Combined with Stochastic Local Search for Feature Selection

verfasst von: Messaouda Nekkaa, Dalila Boughaci

Erschienen in: Neural Processing Letters | Ausgabe 1/2016

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Feature selection is a challenging task that has been the subject of a large amount of research, especially in relation to classification tasks. It permits to eliminate the redundant attributes and enhance the classification accuracy by keeping only the relevant attributes. In this paper, we propose a hybrid search method based on both harmony search algorithm (HSA) and stochastic local search (SLS) for feature selection in data classification. A novel probabilistic selection strategy is used in HSA–SLS to select the appropriate solutions to undergo stochastic local refinement, keeping a good compromise between exploration and exploitation. In addition, the HSA–SLS is combined with a support vector machine (SVM) classifier with optimized parameters. The proposed HSA–SLS method tries to find a subset of features that maximizes the classification accuracy rate of SVM. Experimental results show good performance in favor of our proposed method.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Alia OM, Mandava R, Ramachandram D, Aziz ME (2009) Dynamic fuzzy clustering using harmony search with application to image segmentation. In: IEEE international symposium on signal processing and information technology (ISSPIT), pp 538–543 Alia OM, Mandava R, Ramachandram D, Aziz ME (2009) Dynamic fuzzy clustering using harmony search with application to image segmentation. In: IEEE international symposium on signal processing and information technology (ISSPIT), pp 538–543
2.
Zurück zum Zitat Awadallah MA, Khader AT, Azmi Al-Betar M, Bolaji AL (2013) Global best Harmony Search with a new pitch adjustment designed for Nurse Rostering. J King Saud Univ Comput Inf Sci 25(2):145–162 Awadallah MA, Khader AT, Azmi Al-Betar M, Bolaji AL (2013) Global best Harmony Search with a new pitch adjustment designed for Nurse Rostering. J King Saud Univ Comput Inf Sci 25(2):145–162
3.
Zurück zum Zitat Bao Y, Hu Z, Xiong T (2013) A PSO and pattern search based memetic algorithm for SVMs parameters optimization. Neurocomputing 117:98–106CrossRef Bao Y, Hu Z, Xiong T (2013) A PSO and pattern search based memetic algorithm for SVMs parameters optimization. Neurocomputing 117:98–106CrossRef
4.
Zurück zum Zitat Bermejo P, Gomez JA, Puerta JM (2011) A GRASP algorithm for fast hybrid (filter-wrapper) feature subset selection in high-dimensional datasets. Pattern Recogn Lett 32(5):701–711CrossRef Bermejo P, Gomez JA, Puerta JM (2011) A GRASP algorithm for fast hybrid (filter-wrapper) feature subset selection in high-dimensional datasets. Pattern Recogn Lett 32(5):701–711CrossRef
5.
Zurück zum Zitat Bonilla Huerta EB, Duval B, Hao JK (2006) A hybrid GA/SVM approach for gene selection and classification of microarray data. In: Rothlanf F et al (eds) EvoWorkshops 2006, LNCS, vol 3907, pp 34–44 Bonilla Huerta EB, Duval B, Hao JK (2006) A hybrid GA/SVM approach for gene selection and classification of microarray data. In: Rothlanf F et al (eds) EvoWorkshops 2006, LNCS, vol 3907, pp 34–44
6.
Zurück zum Zitat Boughaci D, Benhamou B, Drias H (2010) Local Search Methods for the optimal winner determination problem. J Math Model Algorithms (Springer) 9(2):165–180MathSciNetCrossRef Boughaci D, Benhamou B, Drias H (2010) Local Search Methods for the optimal winner determination problem. J Math Model Algorithms (Springer) 9(2):165–180MathSciNetCrossRef
7.
Zurück zum Zitat Breiman L, Friedman J, Olshen R, Stone C (1984) Classification and regression trees. Wadsworth, BelmontMATH Breiman L, Friedman J, Olshen R, Stone C (1984) Classification and regression trees. Wadsworth, BelmontMATH
8.
Zurück zum Zitat Campbell C, Ying Y (2011) Learning with support vector machines. Morgan and Claypool, San RafaelMATH Campbell C, Ying Y (2011) Learning with support vector machines. Morgan and Claypool, San RafaelMATH
10.
Zurück zum Zitat Dia R, Shen Q (2012) Feature selection with harmony search. IEEE Trans Syst Man Cybern B 42(6):1509–1523CrossRef Dia R, Shen Q (2012) Feature selection with harmony search. IEEE Trans Syst Man Cybern B 42(6):1509–1523CrossRef
11.
Zurück zum Zitat Frank E, Witten IH (1998) Generating accurate rule sets without global optimization. In: Shavlik J (ed) Proceedings of the fifteenth international conference machine learning (ICML 98), pp 144–151 Frank E, Witten IH (1998) Generating accurate rule sets without global optimization. In: Shavlik J (ed) Proceedings of the fifteenth international conference machine learning (ICML 98), pp 144–151
12.
Zurück zum Zitat Friedman N, Geiger D, Goldszmidt M (1997) Bayesian network classifiers. Mach Learn 29:131–163CrossRefMATH Friedman N, Geiger D, Goldszmidt M (1997) Bayesian network classifiers. Mach Learn 29:131–163CrossRefMATH
13.
Zurück zum Zitat Gao XZ, Wang X, Zenger K (2013) A memetic-inspired harmony search method in optimal wind generator design. In: International journal of machine learning and cybernetics. Springer, Berlin Gao XZ, Wang X, Zenger K (2013) A memetic-inspired harmony search method in optimal wind generator design. In: International journal of machine learning and cybernetics. Springer, Berlin
14.
Zurück zum Zitat Geem ZW (2007) Harmony search algorithm for solving Sudoku. Knowl Based Intell Inf Eng Syst 4692:371–378 Geem ZW (2007) Harmony search algorithm for solving Sudoku. Knowl Based Intell Inf Eng Syst 4692:371–378
15.
Zurück zum Zitat Geem ZW (2009) Harmony search algorithms for structural design optimization. Springer, New YorkCrossRef Geem ZW (2009) Harmony search algorithms for structural design optimization. Springer, New YorkCrossRef
16.
Zurück zum Zitat Geem ZW, Choi JY (2007) Music composition using harmony search algorithm. In: Applications of evolutionary computing. Springer, Berlin, pp 593–600 Geem ZW, Choi JY (2007) Music composition using harmony search algorithm. In: Applications of evolutionary computing. Springer, Berlin, pp 593–600
17.
Zurück zum Zitat Geem ZW, Kim JH, Loganathan G (2001) A new heuristic optimization algorithm: harmony search. Simulation 76:60–68CrossRef Geem ZW, Kim JH, Loganathan G (2001) A new heuristic optimization algorithm: harmony search. Simulation 76:60–68CrossRef
18.
Zurück zum Zitat Hadwan M, Ayob M, Sabar NR, Qu R (2013) A harmony search algorithm for nurse rostering problems. Inf Sci 233:126–140MathSciNetCrossRef Hadwan M, Ayob M, Sabar NR, Qu R (2013) A harmony search algorithm for nurse rostering problems. Inf Sci 233:126–140MathSciNetCrossRef
19.
Zurück zum Zitat Hamel L (2009) Knowledge discovery with support vector machines. Wiley, Canada Hamel L (2009) Knowledge discovery with support vector machines. Wiley, Canada
20.
Zurück zum Zitat Han J, Kamber M (2006) Data mining concepts and techniques, 2nd edn. Morgan Kaufmann, San FranciscoMATH Han J, Kamber M (2006) Data mining concepts and techniques, 2nd edn. Morgan Kaufmann, San FranciscoMATH
21.
Zurück zum Zitat Hannah IH, Bagyamathi M, Azar TA (2015) A novel hybrid feature selection method based on rough set and improved harmony search. Neural Comput Appl. doi:10.1007/s00521-015-1840-0 Hannah IH, Bagyamathi M, Azar TA (2015) A novel hybrid feature selection method based on rough set and improved harmony search. Neural Comput Appl. doi:10.​1007/​s00521-015-1840-0
22.
Zurück zum Zitat Hertz JA, Krogh A, Palmer RG (1991) Introduction to the theory of neural computation. Addison-Wesley Publishing Company, Inc., Redwood City Hertz JA, Krogh A, Palmer RG (1991) Introduction to the theory of neural computation. Addison-Wesley Publishing Company, Inc., Redwood City
24.
Zurück zum Zitat John GH, Langley P (1995) Estimating continuous distributions in Bayesian classifiers. In: Proceedings of the eleventh conference on uncertainty in artificial intelligence. Morgan Kaufmann, San Mateo, pp 338–345 John GH, Langley P (1995) Estimating continuous distributions in Bayesian classifiers. In: Proceedings of the eleventh conference on uncertainty in artificial intelligence. Morgan Kaufmann, San Mateo, pp 338–345
25.
Zurück zum Zitat Kecman V (2001) Learning and soft computing: support vector machines, neural networks, and fuzzy logic models. The MIT Press, LondonMATH Kecman V (2001) Learning and soft computing: support vector machines, neural networks, and fuzzy logic models. The MIT Press, LondonMATH
26.
Zurück zum Zitat Kohavi R, John G (1996) Wrappers for feature subset selection. Artif intell 97(1–2):273–324 (special issue on relevance) Kohavi R, John G (1996) Wrappers for feature subset selection. Artif intell 97(1–2):273–324 (special issue on relevance)
27.
Zurück zum Zitat Krishnaveni V, Arumugam G (2013) Harmony search based wrapper feature selection method for 1-nearest neighbour classifier. In: Proceedings of the international conference on pattern recognition, informatics and mobile engineering (PRIME) Krishnaveni V, Arumugam G (2013) Harmony search based wrapper feature selection method for 1-nearest neighbour classifier. In: Proceedings of the international conference on pattern recognition, informatics and mobile engineering (PRIME)
29.
Zurück zum Zitat Lee KS, Geem ZW (2005) A new meta-heuristic algorithm for continuous engineering optimization: harmony search theory and practice. Comput Methods Appl Mech Eng 194:3902–3922CrossRefMATH Lee KS, Geem ZW (2005) A new meta-heuristic algorithm for continuous engineering optimization: harmony search theory and practice. Comput Methods Appl Mech Eng 194:3902–3922CrossRefMATH
30.
Zurück zum Zitat Lessmann S, Stahlbock R, Crone SF (2006) Genetic algorithms for support vector machine model selection. In: Proceedings of the international joint conference on neural networks, IJCNN 2006, part of the IEEE world congress on computational intelligence, WCCI 2006. IEEE, Vancouver, pp 3063–3069 Lessmann S, Stahlbock R, Crone SF (2006) Genetic algorithms for support vector machine model selection. In: Proceedings of the international joint conference on neural networks, IJCNN 2006, part of the IEEE world congress on computational intelligence, WCCI 2006. IEEE, Vancouver, pp 3063–3069
31.
Zurück zum Zitat Li Y, Tong Y, Bai B, Zhang Y (2007) An improved particle swarm optimization for SVM training. In: Third international conference on natural computation (ICNC 2007), pp 611–615 Li Y, Tong Y, Bai B, Zhang Y (2007) An improved particle swarm optimization for SVM training. In: Third international conference on natural computation (ICNC 2007), pp 611–615
32.
Zurück zum Zitat Mahdavi M, Fesanghary M, Damangir E (2007) An improved harmony search algorithm for solving optimization problems. Appl Math Comput 188(2):1567–1579MathSciNetMATH Mahdavi M, Fesanghary M, Damangir E (2007) An improved harmony search algorithm for solving optimization problems. Appl Math Comput 188(2):1567–1579MathSciNetMATH
33.
Zurück zum Zitat Nekkaa M, Boughaci D (2012) Improving support vector machine using a stochastic local search for classification in data mining. In: Proceedings of ICONIP 2012, Part II, LNCS proceedings, vol 7664, pp 168–176 Nekkaa M, Boughaci D (2012) Improving support vector machine using a stochastic local search for classification in data mining. In: Proceedings of ICONIP 2012, Part II, LNCS proceedings, vol 7664, pp 168–176
34.
Zurück zum Zitat Nekkaa M, Boughaci D (2014) Stochastic local search versus genetic algorithm for feature selection. In: Proceedings of APMOD CONFERENCE 2014: international conference on applied mathematical optimization and modelling 2014 Nekkaa M, Boughaci D (2014) Stochastic local search versus genetic algorithm for feature selection. In: Proceedings of APMOD CONFERENCE 2014: international conference on applied mathematical optimization and modelling 2014
36.
Zurück zum Zitat Nekooei K, Farsangi MM, Nezamabadi-Pour H, Lee KY (2013) An improved multi-objective harmony search for optimal placement of DGs in distribution systems. IEEE Trans Smart Grid 4(1):557–567CrossRef Nekooei K, Farsangi MM, Nezamabadi-Pour H, Lee KY (2013) An improved multi-objective harmony search for optimal placement of DGs in distribution systems. IEEE Trans Smart Grid 4(1):557–567CrossRef
37.
Zurück zum Zitat Panchal A (2009) Harmony search in therapeutic medical physics. In: Geem ZW (ed) Music-inspired harmony search algorithm. Springer, Hiedelberg, pp 189–203 Panchal A (2009) Harmony search in therapeutic medical physics. In: Geem ZW (ed) Music-inspired harmony search algorithm. Springer, Hiedelberg, pp 189–203
38.
Zurück zum Zitat Quinlan JR (1992) C4.5: programs for machine learning. Morgan Kaufmann, San Mateo Quinlan JR (1992) C4.5: programs for machine learning. Morgan Kaufmann, San Mateo
39.
Zurück zum Zitat Rao R, Savsani V, Vakharia D (2012) Teaching learning-based optimization: an optimization method for continuous non-linear large scale problems. Inf Sci 183:1–15MathSciNetCrossRef Rao R, Savsani V, Vakharia D (2012) Teaching learning-based optimization: an optimization method for continuous non-linear large scale problems. Inf Sci 183:1–15MathSciNetCrossRef
40.
Zurück zum Zitat Tan KC, Teoh EJ, Yua Q, Goh KC (2009) A hybrid evolutionary algorithm for attribute selection in data mining. Expert Syst Appl 36:8616–8630CrossRef Tan KC, Teoh EJ, Yua Q, Goh KC (2009) A hybrid evolutionary algorithm for attribute selection in data mining. Expert Syst Appl 36:8616–8630CrossRef
41.
Zurück zum Zitat Tay FEH, Cao LJ (2001) Application of support vector machines in financial time series forecasting. Omega 29(4):309–317CrossRef Tay FEH, Cao LJ (2001) Application of support vector machines in financial time series forecasting. Omega 29(4):309–317CrossRef
42.
Zurück zum Zitat Vapnik V (1998) Statistical learning theory. Wiley, New YorkMATH Vapnik V (1998) Statistical learning theory. Wiley, New YorkMATH
43.
45.
Zurück zum Zitat Yadav P, Kumar R, Panda S, Chang C (2012) An intelligent tuned harmony search algorithm for optimisation. Inf Sci 196:47–72CrossRef Yadav P, Kumar R, Panda S, Chang C (2012) An intelligent tuned harmony search algorithm for optimisation. Inf Sci 196:47–72CrossRef
46.
Zurück zum Zitat Yildiz AR (2012) A comparative study of population-based optimization algorithms for turning operations. Inf Sci 210:81–88MathSciNetCrossRef Yildiz AR (2012) A comparative study of population-based optimization algorithms for turning operations. Inf Sci 210:81–88MathSciNetCrossRef
Metadaten
Titel
Hybrid Harmony Search Combined with Stochastic Local Search for Feature Selection
verfasst von
Messaouda Nekkaa
Dalila Boughaci
Publikationsdatum
01.08.2016
Verlag
Springer US
Erschienen in
Neural Processing Letters / Ausgabe 1/2016
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-015-9450-5

Weitere Artikel der Ausgabe 1/2016

Neural Processing Letters 1/2016 Zur Ausgabe

Neuer Inhalt