Skip to main content
SpringerLink
Log in

Artificial neural network regression as a local search heuristic for ensemble strategies in differential evolution

  • Original Paper
  • Published:
Nonlinear Dynamics Aims and scope Submit manuscript

Abstract

Nature frequently serves as an inspiration for developing new algorithms to solve challenging real-world problems. Mathematical modeling has led to the development of artificial neural networks (ANNs), which have proven especially useful for solving problems such as classification and regression. Moreover, evolutionary algorithms (EAs), inspired by Darwin’s natural evolution, have been successfully applied to solve optimization, modeling, and simulation problems. Differential evolution (DE) is a particularly well-known EA that possesses a multitude of strategies for generating an offspring solution, where the best strategy is not known in advance. In this paper, the ANN regression is applied as a local search heuristic within the DE algorithm that tries predicting the best strategy or attempting to generate a better offspring from an ensemble of DE strategies. This local search heuristic is applied to the population of solutions according to a control parameter that regulates between the time complexity of calculation and the quality of the solution. The experiments on a CEC 2014 test suite consisting of 30 benchmark functions reveal the full potential in developing this idea.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Adeyemo, J., Otieno, F.: Differential evolution algorithm for solving multi-objective crop planning model. Agric. Water Manag. 97(6), 848–856 (2010)

    Article  Google Scholar 

  2. Bhattacharya, A., Chattopadhyay, P.K.: Solving economic emission load dispatch problems using hybrid differential evolution. Appl. Soft Comput. 11(2), 2526–2537 (2011)

    Article  Google Scholar 

  3. Bigus, J.P.: Data Mining with Neural Networks: Solving Business Problems from Application Development to Decision Support. McGraw-Hill, Inc., New York (1996)

    Google Scholar 

  4. Brest, J., Greiner, S., Boskovic, B., Mernik, M., Zumer, V.: Self-adapting control parameters in differential evolution: a comparative study on numerical benchmark problems. IEEE Trans. Evolut. Comput. 10(6), 646–657 (2006)

    Article  Google Scholar 

  5. Cai, Y., Wang, J.: Differential evolution with neighborhood and direction information for numerical optimization. IEEE Trans. Cybern. 43(6), 2202–2215 (2013)

    Article  Google Scholar 

  6. Chakraborty, U.K., Das, S., Konar, A.: Differential evolution with local neighborhood. In: IEEE Congress on Evolutionary Computation, pp. 2042–2049. IEEE (2006)

  7. Darwin, C.: On the Origin of Species. Harvard University Press, London (1859)

    Google Scholar 

  8. Das, S., Suganthan, P.: Differential evolution: a survey of the state-of-the-art. IEEE Trans. Evolut. Comput. 15(1), 4–31 (2011)

    Article  Google Scholar 

  9. Das, S., Abraham, A., Chakraborty, U.K., Konar, A.: Differential evolution using a neighborhood-based mutation operator. IEEE Trans. Evolut. Comput. 13(3), 526–553 (2009)

    Article  Google Scholar 

  10. Datta, D., Dutta, S.: A binary-real-coded differential evolution for unit commitment problem. Int. J. Electr. Power Energy Syst. 42(1), 517–524 (2012)

    Article  Google Scholar 

  11. Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)

    MathSciNet  MATH  Google Scholar 

  12. Elsayed, S.M., Sarker, R.A., Essam, D.L.: Multi-operator based evolutionary algorithms for solving constrained optimization problems. Comput. Oper. Res. 38(12), 1877–1896 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  13. Fan, H.Y., Lampinen, J.: A trigonometric mutation operation to differential evolution. J. Global Optim. 27(1), 105–129 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  14. Fister, I.J., Suganthan, P.N., Strnad, D., Brest, J., Fister, I.: Artificial neural network regression on ensemble strategies in differential evolution. In: MENDEL 2014, 20th International Conference on Soft Computing. University of Technology, Brno (2014)

  15. Fister, I., Fister Jr, I., Yang, X.S., Brest, J.: A comprehensive review of firefly algorithms. Swarm Evolut. Comput. 13, 34–46 (2013)

    Article  Google Scholar 

  16. Fister, I., Rauter, S., Yang, X.S., Ljubič, K., Fister Jr, I.: Planning the sports training sessions with the bat algorithm. Neurocomputing 149(Part B), 993–1002 (2015)

    Article  Google Scholar 

  17. Fister Jr, I., Fister, D., Fister, I.: Differential evolution strategies with random forest regression in the bat algorithm. In: Proceeding of the Fifteenth Annual Conference Companion on Genetic and Evolutionary Computation Conference Companion 2013, pp. 1703–1706. ACM (2013)

  18. Fister Jr, I., Fister, D., Yang, X.S.: A hybrid bat algorithm. Elektroteh. vestnik 80(1–2), 1–7 (2013)

    MATH  Google Scholar 

  19. Friedman, M.: A comparison of alternative tests of significance for the problem of m rankings. Ann. Math. Stat. 11, 86–92 (1940)

    Article  MathSciNet  MATH  Google Scholar 

  20. Garro, B.A., Sossa, H., Vázquez, R.A.: Design of artificial neural networks using differential evolution algorithm. In: Wong, K.W., Mendis, B.S.U., Bouzerdoum, A. (eds.) Neural Information Processing. Models and Applications, pp. 201–208. Springer, Heidelberg (2010)

  21. Gershenson, C.: Artificial Neural Networks for Beginners (2003). arXiv:cs/0308031

  22. Han, M.F., Liao, S.H., Chang, J.Y., Lin, C.T.: Dynamic group-based differential evolution using a self-adaptive strategy for global optimization problems. Appl. Intell. 39(1), 41–56 (2013)

    Article  Google Scholar 

  23. Hecht-Nielsen, R.: Theory of the backpropagation neural network. In: International Joint Conference on Neural Networks, pp. 593–605. IEEE (1989)

  24. Holger, H., Thomas, S.: Stochastic Local Search: Foundations & Applications. Morgan Kaufman Inc., Amsterdam (2004)

    MATH  Google Scholar 

  25. Hozjan, T., Turk, G., Srpčič, S.: Fire analysis of steel frames with the use of artificial neural networks. J. Constr. Steel Res. 63(10), 1396–1403 (2007)

    Article  Google Scholar 

  26. J.J. Liang, B.Y.Q., Suganthan, P.N.: Problem definitions and evaluation criteria for the CEC 2014 special session and competition on single objective real-parameter numerical optimization. Technical Report, Zhengzhou University and Nanyang Technological University (2013)

  27. Kartam, N., Flood, I., Garrett, J.H.: Artificial Neural Networks for Civil Engineers: Fundamentals and Applications. American Society of Civil Engineers, New York (1997)

    Google Scholar 

  28. Kourentzes, N., Barrow, D.K., Crone, S.F.: Neural network ensemble operators for time series forecasting. Expert Syst. Appl. 41(9), 4235–4244 (2014)

    Article  Google Scholar 

  29. LaTorre, A., Muelas, S., Peña, J.M.: A mos-based dynamic memetic differential evolution algorithm for continuous optimization: a scalability test. Soft Comput. 15(11), 2187–2199 (2011)

    Article  Google Scholar 

  30. Lee, S., Choeh, J.Y.: Predicting the helpfulness of online reviews using multilayer perceptron neural networks. Expert Syst. Appl. 41(6), 3041–3046 (2014)

    Article  Google Scholar 

  31. Lin, Y.C., Hwang, K.S., Wang, F.S.: Co-evolutionary hybrid differential evolution for mixed-integer optimization problems. Eng. Optim. 33(6), 663–682 (2001)

    Article  Google Scholar 

  32. Mallipeddi, R., Suganthan, P.: Differential evolution algorithm with ensemble of populations for global numerical optimization. Opsearch 46(2), 184–213 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  33. Mallipeddi, R., Mallipeddi, S., Suganthan, P.N.: Ensemble strategies with adaptive evolutionary programming. Inf. Sci. 180(9), 1571–1581 (2010)

    Article  MATH  Google Scholar 

  34. Mallipeddi, R., Suganthan, P.N., Pan, Q.K., Tasgetiren, M.F.: Differential evolution algorithm with ensemble of parameters and mutation strategies. Appl. Soft Comput. 11(2), 1679–1696 (2011)

    Article  Google Scholar 

  35. Mallipeddi, R., Suganthan, P.N.: Differential evolution algorithm with ensemble of parameters and mutation and crossover strategies. In: Panigrahi, B.K., Das, S., Suganthan, P.N., Dash, S.S. (eds.) Swarm, Evolutionary, and Memetic Computing, pp. 71–78. Springer, Heidelberg (2010)

  36. Mallipeddi, R., Suganthan, P.N.: Ensemble differential evolution algorithm for CEC2011 problems. In: IEEE Congress on Evolutionary Computation, pp. 1557–1564. IEEE (2011)

  37. McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5(4), 115–133 (1943)

    Article  MathSciNet  MATH  Google Scholar 

  38. Mininno, E., Neri, F., Cupertino, F., Naso, D.: Compact differential evolution. IEEE Trans. Evolut. Comput. 15(1), 32–54 (2011)

    Article  Google Scholar 

  39. Neri, F., Iacca, G., Mininno, E.: Disturbed exploitation compact differential evolution for limited memory optimization problems. Inf. Sci. 181(12), 2469–2487 (2011)

    Article  MathSciNet  Google Scholar 

  40. Neri, F., Mininno, E.: Memetic compact differential evolution for Cartesian robot control. IEEE Comput. Intell. Mag. 5(2), 54–65 (2010)

    Article  Google Scholar 

  41. Piotrowski, A.P.: Adaptive memetic differential evolution with global and local neighborhood-based mutation operators. Inf. Sci. 241, 164–194 (2013)

    Article  Google Scholar 

  42. Qin, A.K., Suganthan, P.N.: Self-adaptive differential evolution algorithm for numerical optimization. In: IEEE Congress on Evolutionary Computation, vol. 2, pp. 1785–1791. IEEE (2005)

  43. Rahnamayan, S., Tizhoosh, H.R., Salama, M.M.: Opposition-based differential evolution. IEEE Trans. Evolut. Comput. 12(1), 64–79 (2008)

    Article  Google Scholar 

  44. Rocca, P., Oliveri, G., Massa, A.: Differential evolution as applied to electromagnetics. IEEE Antennas Propag. Mag. 53(1), 38–49 (2011)

    Article  Google Scholar 

  45. Rojas, R.: Neutral Networks: A Systematic Introduction. Springer, Berlin (1996)

  46. Russell, S.J., Norvig, P.: Artificial Intelligence: A Modern Approach, 3rd edn. Prentice Hall, Englewood Cliffs (2010)

    MATH  Google Scholar 

  47. Santos, J., Diéguez, M.: Differential evolution for protein structure prediction using the HP model. In: Ferrández, J.M., Álvarez, J.R., de la Paz, F., Toledo, F.J. (eds.) Foundations on Natural and Artificial Computation, pp. 323–333. Springer, Heidelberg (2011)

  48. Storn, R., Price, K.: Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11(4), 341–359 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  49. Tvrdìk, J.: Competitive differential evolution. In: MENDEL 2006. 12th international conference on soft computing, pp. 7–12. University of Technology, Brno (2006)

  50. Tvrdìk, J.: Differential evolution with competitive setting of its control parameters. TASK Q. 11, 169–179 (2007)

    Google Scholar 

  51. Tvrdìk, J.: Adaptation in differential evolution: a numerical comparison. Appl. Soft Comput. 9, 1149–1155 (2009)

    Article  Google Scholar 

  52. Uyar, A.Ş., Türkay, B., Keleş, A.: A novel differential evolution application to short-term electrical power generation scheduling. Int. J. Electr. Power Energy Syst. 33(6), 1236–1242 (2011)

    Article  Google Scholar 

  53. Vrugt, J.A., Robinson, B.A., Hyman, J.M.: Self-adaptive multimethod search for global optimization in real-parameter spaces. IEEE Trans. Evolut. Comput. 13(2), 243–259 (2009)

    Article  Google Scholar 

  54. Widrow, B., Rumelhart, D.E., Lehr, M.A.: Neural networks: applications in industry, business and science. Commun. ACM 37(3), 93–105 (1994)

    Article  Google Scholar 

  55. Zhang, W.J., Xie, X.F., et al.: DEPSO: hybrid particle swarm with differential evolution operator. IEEE Int. Conf. Syst. Man Cybern. 4, 3816–3821 (2003)

    Google Scholar 

  56. Zobaa, A., Reljin, B.: Neural network applications in electrical engineering. Neurocomputing 70(16–18), 2613–2614 (2007)

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported by the Slovenian Research Agency (Grant P5-0027) and by the Deanship of Scientific Research, King Abdulaziz University (Grant 76-130-35-HiCi).

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering and Computer Science, University of Maribor, Smetanova 17, 2000, Maribor, Slovenia

    Iztok Fister, Iztok Fister Jr. & Damjan Strnad

  2. School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore

    Ponnuthurai Nagaratnam Suganthan

  3. Department of Physics, Faculty of Sciences, King Abdulaziz University, Jeddah, Saudi Arabia

    Salahuddin M. Kamal, Fahad M. Al-Marzouki & Matjaž Perc

  4. Faculty of Natural Sciences and Mathematics, University of Maribor, Koroška cesta 160, 2000, Maribor, Slovenia

    Matjaž Perc

Authors
  1. Iztok Fister
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ponnuthurai Nagaratnam Suganthan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Iztok Fister Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Salahuddin M. Kamal
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fahad M. Al-Marzouki
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Matjaž Perc
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Damjan Strnad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matjaž Perc.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fister, I., Suganthan, P.N., Fister, I. et al. Artificial neural network regression as a local search heuristic for ensemble strategies in differential evolution. Nonlinear Dyn 84, 895–914 (2016). https://doi.org/10.1007/s11071-015-2537-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11071-015-2537-8

Keywords

Search

Navigation