Skip to main content
Erschienen in: Soft Computing 2/2012

01.02.2012 | Original Paper

Learning-enhanced differential evolution for numerical optimization

verfasst von: Yiqiao Cai, Jiahai Wang, Jian Yin

Erschienen in: Soft Computing | Ausgabe 2/2012

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Differential evolution (DE) is a simple and powerful population-based search algorithm, successfully used in various scientific and engineering fields. However, DE is not free from the problems of stagnation and premature convergence. Hence, designing more effective search strategies to enhance the performance of DE is one of the most salient and active topics. This paper proposes a new method, called learning-enhanced DE (LeDE) that promotes individuals to exchange information systematically. Distinct from the existing DE variants, LeDE adopts a novel learning strategy, namely clustering-based learning strategy (CLS). In CLS, there are two levels of learning strategies, intra-cluster learning strategy and inter-cluster learning strategy. They are adopted for exchanging information within the same cluster and between different clusters, respectively. Experimental studies over 23 benchmark functions show that LeDE significantly outperforms the conventional DE. Compared with other clustering-based DE algorithms, LeDE can obtain better solutions. In addition, LeDE is also shown to be significantly better than or at least comparable to several state-of-art DE variants as well as some other evolutionary algorithms.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Fußnoten
1
The upper bound of the number of clusters is set to \(\sqrt{{\rm NP}}, \) which is a rule of thumb used in many researches (Sheng et al. 2005).
 
2
Since the CEC2005 test functions are defined up to D = 50 in Suganthan et al. (2005), they are tested only at D = 30 and 50 in this paper.
 
Literatur
Zurück zum Zitat Alessandro P, Antonina S (2008) Particle swarm optimization for multimodal functions: a clustering approach. J Artif Evol Appl 2008:1–15 Alessandro P, Antonina S (2008) Particle swarm optimization for multimodal functions: a clustering approach. J Artif Evol Appl 2008:1–15
Zurück zum Zitat Brest J, Greiner S, Boskovic B, Mernik M, Zumer V (2006) Self-adapting control parameters in differential evolution: a comparative study on numerical benchmark problems. IEEE Trans Evol Comput 10(6):646–657CrossRef Brest J, Greiner S, Boskovic B, Mernik M, Zumer V (2006) Self-adapting control parameters in differential evolution: a comparative study on numerical benchmark problems. IEEE Trans Evol Comput 10(6):646–657CrossRef
Zurück zum Zitat Cai Z, Gong W, Ling C, Zhang H (2011) A clustering-based differential evolution for global optimization. Appl Soft Comput 11(1):1363–1379CrossRef Cai Z, Gong W, Ling C, Zhang H (2011) A clustering-based differential evolution for global optimization. Appl Soft Comput 11(1):1363–1379CrossRef
Zurück zum Zitat Damavandi N, Safavi-Naeini S (2005) A hybrid evolutionary programming method for circuit optimization. IEEE Trans Circuits Syst I Regul Pap 52(5):902–910MathSciNetCrossRef Damavandi N, Safavi-Naeini S (2005) A hybrid evolutionary programming method for circuit optimization. IEEE Trans Circuits Syst I Regul Pap 52(5):902–910MathSciNetCrossRef
Zurück zum Zitat Das S, Abraham A, Chakraborty U, Konar A (2009) Differential evolution using a neighborhood-based mutation operator. IEEE Trans Evol Comput 13(3):526–553CrossRef Das S, Abraham A, Chakraborty U, Konar A (2009) Differential evolution using a neighborhood-based mutation operator. IEEE Trans Evol Comput 13(3):526–553CrossRef
Zurück zum Zitat Das S, Suganthan PN (2011) Differential evolution: a survey of the state-of-the-art. IEEE Trans Evol Comput 15(1):4–13CrossRef Das S, Suganthan PN (2011) Differential evolution: a survey of the state-of-the-art. IEEE Trans Evol Comput 15(1):4–13CrossRef
Zurück zum Zitat Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1(1):3–18CrossRef Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1(1):3–18CrossRef
Zurück zum Zitat Emmendorfer L, Pozo A (2009) Effective linkage learning using low-order statistics and clustering. IEEE Trans Evol Comput 13(6):1233–1246CrossRef Emmendorfer L, Pozo A (2009) Effective linkage learning using low-order statistics and clustering. IEEE Trans Evol Comput 13(6):1233–1246CrossRef
Zurück zum Zitat García S, Herrera F (2008) An extension on “statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. J Mach Learn Res 9:2677–2694MATH García S, Herrera F (2008) An extension on “statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. J Mach Learn Res 9:2677–2694MATH
Zurück zum Zitat García S, Fernández A, Luengo J, Herrera F (2009a) A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability. Soft Comput Fusion Found Methodol Appl 13 (10):959–977 García S, Fernández A, Luengo J, Herrera F (2009a) A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability. Soft Comput Fusion Found Methodol Appl 13 (10):959–977
Zurück zum Zitat García S, Molina D, Lozano M, Herrera F (2009b) A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’ 2005 special session on real parameter optimization. J Heuristics 15(6):617–644MATHCrossRef García S, Molina D, Lozano M, Herrera F (2009b) A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’ 2005 special session on real parameter optimization. J Heuristics 15(6):617–644MATHCrossRef
Zurück zum Zitat García S, Fernández A, Luengo J, Herrera F (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power. Inf Sci 180 (10):2044–2064CrossRef García S, Fernández A, Luengo J, Herrera F (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power. Inf Sci 180 (10):2044–2064CrossRef
Zurück zum Zitat Ilonen J, Kamarainen J, Lampinen J (2003) Differential evolution training algorithm for feed-forward neural networks. Neural Process Lett 17(1):93–105CrossRef Ilonen J, Kamarainen J, Lampinen J (2003) Differential evolution training algorithm for feed-forward neural networks. Neural Process Lett 17(1):93–105CrossRef
Zurück zum Zitat Iorio A, Li X (2011) Improving the performance and scalability of differential evolution on problems exhibiting parameter interactions. Soft Comput Fusion Found Methodol Appl. doi:10.1007/s00500-010-0614-y Iorio A, Li X (2011) Improving the performance and scalability of differential evolution on problems exhibiting parameter interactions. Soft Comput Fusion Found Methodol Appl. doi:10.​1007/​s00500-010-0614-y
Zurück zum Zitat Jain A, Murty M, Flynn P (1999) Data clustering: a review. ACM Comput Surv (CSUR) 31(3):264–323CrossRef Jain A, Murty M, Flynn P (1999) Data clustering: a review. ACM Comput Surv (CSUR) 31(3):264–323CrossRef
Zurück zum Zitat Joshi R, Sanderson A (1999) Minimal representation multisensor fusion using differential evolution. IEEE Trans Syst Man Cybern Part A Syst Hum 29(1):63–76CrossRef Joshi R, Sanderson A (1999) Minimal representation multisensor fusion using differential evolution. IEEE Trans Syst Man Cybern Part A Syst Hum 29(1):63–76CrossRef
Zurück zum Zitat Kennedy J (2000) Stereotyping: improving particle swarm performance with cluster analysis. In: Proceedings of the IEEE congress on evolutionary computation, California, USA, pp 1507–1512 Kennedy J (2000) Stereotyping: improving particle swarm performance with cluster analysis. In: Proceedings of the IEEE congress on evolutionary computation, California, USA, pp 1507–1512
Zurück zum Zitat Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of IEEE international conference on neural networks (ICNN’95), pp 1942–1948 Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of IEEE international conference on neural networks (ICNN’95), pp 1942–1948
Zurück zum Zitat Lee C, Yao X (2004) Evolutionary programming using mutations based on the Lévy probability distribution. IEEE Trans Evol Comput 8(1):1–13CrossRef Lee C, Yao X (2004) Evolutionary programming using mutations based on the Lévy probability distribution. IEEE Trans Evol Comput 8(1):1–13CrossRef
Zurück zum Zitat Leung Y, Wang Y (2001) An orthogonal genetic algorithm with quantization for global numerical optimization. IEEE Transa Evol Comput 5(1):41–53CrossRef Leung Y, Wang Y (2001) An orthogonal genetic algorithm with quantization for global numerical optimization. IEEE Transa Evol Comput 5(1):41–53CrossRef
Zurück zum Zitat Li M, Kou J (2008) Crowding with nearest neighbors replacement for multiple species niching and building blocks preservation in binary multimodal functions optimization. J Heuristics 14 (3):243–270MATHCrossRef Li M, Kou J (2008) Crowding with nearest neighbors replacement for multiple species niching and building blocks preservation in binary multimodal functions optimization. J Heuristics 14 (3):243–270MATHCrossRef
Zurück zum Zitat Liu J, Lampinen J (2005) A fuzzy adaptive differential evolution algorithm. Soft Comput Fusion Found Methodol Appl 9(6):448–462MATH Liu J, Lampinen J (2005) A fuzzy adaptive differential evolution algorithm. Soft Comput Fusion Found Methodol Appl 9(6):448–462MATH
Zurück zum Zitat Lu Q, Yao X (2005) Clustering and learning Gaussian distribution for continuous optimization. IEEE Trans Syst Man Cybern Part C Appl Rev 35(2):195–204CrossRef Lu Q, Yao X (2005) Clustering and learning Gaussian distribution for continuous optimization. IEEE Trans Syst Man Cybern Part C Appl Rev 35(2):195–204CrossRef
Zurück zum Zitat Neri F, Tirronen V (2009) Scale factor local search in differential evolution. Memet Comput 1(2):153–171CrossRef Neri F, Tirronen V (2009) Scale factor local search in differential evolution. Memet Comput 1(2):153–171CrossRef
Zurück zum Zitat Neri F, Tirronen V (2010) Recent advances in differential evolution: a survey and experimental analysis. Artif Intell Rev 33(1):61–106 Neri F, Tirronen V (2010) Recent advances in differential evolution: a survey and experimental analysis. Artif Intell Rev 33(1):61–106
Zurück zum Zitat Noman N, Iba H (2008) Accelerating differential evolution using an adaptive local search. IEEE Trans Evol Comput 12(1):107–125CrossRef Noman N, Iba H (2008) Accelerating differential evolution using an adaptive local search. IEEE Trans Evol Comput 12(1):107–125CrossRef
Zurück zum Zitat Parrott D, Li X (2006) Locating and tracking multiple dynamic optima by a particle swarm model using speciation. IEEE Trans Evol Comput 10(4):440–458CrossRef Parrott D, Li X (2006) Locating and tracking multiple dynamic optima by a particle swarm model using speciation. IEEE Trans Evol Comput 10(4):440–458CrossRef
Zurück zum Zitat Pelikan M, Goldberg D (2000) Genetic algorithms, clustering, and the breaking of symmetry. In: the 6th international conference on parallel problem solving from nature, 2000. Springer, Berlin, pp 385–394 Pelikan M, Goldberg D (2000) Genetic algorithms, clustering, and the breaking of symmetry. In: the 6th international conference on parallel problem solving from nature, 2000. Springer, Berlin, pp 385–394
Zurück zum Zitat Plagianakos V, Tasoulis D, Vrahatis M (2008) A review of major application areas of differential evolution. In: Advances in differential evolution, vol 143, 2008. Springer, Berlin, pp 197–238 Plagianakos V, Tasoulis D, Vrahatis M (2008) A review of major application areas of differential evolution. In: Advances in differential evolution, vol 143, 2008. Springer, Berlin, pp 197–238
Zurück zum Zitat Price K, Storn R, Lampinen J (2005) Differential evolution: a practical approach to global optimization. Springer, New YorkMATH Price K, Storn R, Lampinen J (2005) Differential evolution: a practical approach to global optimization. Springer, New YorkMATH
Zurück zum Zitat Qin AK, Huang VL, Suganthan PN (2009) Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Trans Evol Comput 13 (2):398–417CrossRef Qin AK, Huang VL, Suganthan PN (2009) Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Trans Evol Comput 13 (2):398–417CrossRef
Zurück zum Zitat Rahnamayan S, Tizhoosh HR, Salama MMA (2008) Opposition-based differential evolution. IEEE Trans Evol Comput 12(1):64–79CrossRef Rahnamayan S, Tizhoosh HR, Salama MMA (2008) Opposition-based differential evolution. IEEE Trans Evol Comput 12(1):64–79CrossRef
Zurück zum Zitat Ray T, Liew K (2003) Society and civilization: an optimization algorithm based on the simulation of social behavior. IEEE Trans Evol Comput 7(4):386–396CrossRef Ray T, Liew K (2003) Society and civilization: an optimization algorithm based on the simulation of social behavior. IEEE Trans Evol Comput 7(4):386–396CrossRef
Zurück zum Zitat Sheng W, Swift S, Zhang L, Liu X (2005) A weighted sum validity function for clustering with a hybrid niching genetic algorithm. IEEE Trans Syst Man Cybern Part B Cybern 35(6):1156–1167CrossRef Sheng W, Swift S, Zhang L, Liu X (2005) A weighted sum validity function for clustering with a hybrid niching genetic algorithm. IEEE Trans Syst Man Cybern Part B Cybern 35(6):1156–1167CrossRef
Zurück zum Zitat Storn R, Price K (1997) Differential evolution—a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11(4):341–359MATHMathSciNetCrossRef Storn R, Price K (1997) Differential evolution—a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11(4):341–359MATHMathSciNetCrossRef
Zurück zum Zitat Suganthan P, Hansen N, Liang J, Deb K, Chen Y, Auger A, Tiwari S (2005) Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. Nanyang Technol Universiy, Singapore, pp 1–50 Suganthan P, Hansen N, Liang J, Deb K, Chen Y, Auger A, Tiwari S (2005) Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. Nanyang Technol Universiy, Singapore, pp 1–50
Zurück zum Zitat Sun J, Zhang Q, Tsang EPK (2005) DE/EDA: a new evolutionary algorithm for global optimization. Inf Sci 169 (3–4):249–262MathSciNetCrossRef Sun J, Zhang Q, Tsang EPK (2005) DE/EDA: a new evolutionary algorithm for global optimization. Inf Sci 169 (3–4):249–262MathSciNetCrossRef
Zurück zum Zitat Tirronen V, Neri F, Kärkkäinen T, Majava K, Rossi T (2008) An enhanced memetic differential evolution in filter design for defect detection in paper production. Evol Comput 16 (4):529–555CrossRef Tirronen V, Neri F, Kärkkäinen T, Majava K, Rossi T (2008) An enhanced memetic differential evolution in filter design for defect detection in paper production. Evol Comput 16 (4):529–555CrossRef
Zurück zum Zitat Wang F, Jang H (2000) Parameter estimation of a bioreaction model by hybrid differential evolution. In: Proceedings of 2000 IEEE congress on evolutionary computation, 2000, pp 410–417 Wang F, Jang H (2000) Parameter estimation of a bioreaction model by hybrid differential evolution. In: Proceedings of 2000 IEEE congress on evolutionary computation, 2000, pp 410–417
Zurück zum Zitat Wang Y, Zhang J, Zhang G (2007) A dynamic clustering based differential evolution algorithm for global optimization. Eur J Oper Res 183(1):56–73MATHCrossRef Wang Y, Zhang J, Zhang G (2007) A dynamic clustering based differential evolution algorithm for global optimization. Eur J Oper Res 183(1):56–73MATHCrossRef
Zurück zum Zitat Wong K, Leung K, Wong M (2010) Effect of spatial locality on an evolutionary algorithm for multimodal optimization. In: Applications of evolutionary computation, vol 6024/2010. Springer, Berlin, pp 481–490 Wong K, Leung K, Wong M (2010) Effect of spatial locality on an evolutionary algorithm for multimodal optimization. In: Applications of evolutionary computation, vol 6024/2010. Springer, Berlin, pp 481–490
Zurück zum Zitat Wright A (1991) Genetic algorithms for real parameter optimization. In: Rawlins GJ (ed) Foundations of genetic algorithms, vol 1. Morgan Kaufmann, San Mateo, pp 205–218 Wright A (1991) Genetic algorithms for real parameter optimization. In: Rawlins GJ (ed) Foundations of genetic algorithms, vol 1. Morgan Kaufmann, San Mateo, pp 205–218
Zurück zum Zitat Wu S, Chow T (2007) Self-organizing and self-evolving neurons: a new neural network for optimization. IEEE Trans Neural Netw 18(2):385–396CrossRef Wu S, Chow T (2007) Self-organizing and self-evolving neurons: a new neural network for optimization. IEEE Trans Neural Netw 18(2):385–396CrossRef
Zurück zum Zitat Yang S, Li C (2010) A clustering particle swarm optimizer for locating and tracking multiple optima in dynamic environments. IEEE Trans Evol Comput 14(6):959–974CrossRef Yang S, Li C (2010) A clustering particle swarm optimizer for locating and tracking multiple optima in dynamic environments. IEEE Trans Evol Comput 14(6):959–974CrossRef
Zurück zum Zitat Yang Z, Yao X, He J (2008) Making a difference to differential evolution. In: Advances in metaheuristics for hard optimization. Springer, Berlin, pp 397–414 Yang Z, Yao X, He J (2008) Making a difference to differential evolution. In: Advances in metaheuristics for hard optimization. Springer, Berlin, pp 397–414
Zurück zum Zitat Yao X, Liu Y, Lin G (1999) Evolutionary programming made faster. IEEE Trans Evol Comput 3(2):82–102CrossRef Yao X, Liu Y, Lin G (1999) Evolutionary programming made faster. IEEE Trans Evol Comput 3(2):82–102CrossRef
Zurück zum Zitat Zhang J, Sanderson A (2009) JADE: adaptive differential evolution with optional external archive. IEEE Trans Evol Comput 13(5):945–958CrossRef Zhang J, Sanderson A (2009) JADE: adaptive differential evolution with optional external archive. IEEE Trans Evol Comput 13(5):945–958CrossRef
Zurück zum Zitat Zhong W, Liu J, Xue M, Jiao L (2004) A multiagent genetic algorithm for global numerical optimization. IEEE Trans Syst Man Cybern Part B Cybern 34(2):1128–1141CrossRef Zhong W, Liu J, Xue M, Jiao L (2004) A multiagent genetic algorithm for global numerical optimization. IEEE Trans Syst Man Cybern Part B Cybern 34(2):1128–1141CrossRef
Metadaten
Titel
Learning-enhanced differential evolution for numerical optimization
verfasst von
Yiqiao Cai
Jiahai Wang
Jian Yin
Publikationsdatum
01.02.2012
Verlag
Springer-Verlag
Erschienen in
Soft Computing / Ausgabe 2/2012
Print ISSN: 1432-7643
Elektronische ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-011-0744-x

Weitere Artikel der Ausgabe 2/2012

Soft Computing 2/2012 Zur Ausgabe