Skip to main content
Erschienen in: Neural Computing and Applications 8/2011

01.11.2011 | Original Article

Metaheuristics for the feedforward artificial neural network (ANN) architecture optimization problem

verfasst von: Adenilson R. Carvalho, Fernando M. Ramos, Antonio A. Chaves

Erschienen in: Neural Computing and Applications | Ausgabe 8/2011

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This article deals with evolutionary artificial neural network (ANN) and aims to propose a systematic and automated way to find out a proper network architecture. To this, we adapt four metaheuristics to resolve the problem posed by the pursuit of optimum feedforward ANN architecture and introduced a new criteria to measure the ANN performance based on combination of training and generalization error. Also, it is proposed a new method for estimating the computational complexity of the ANN architecture based on the number of neurons and epochs needed to train the network. We implemented this approach in software and tested it for the problem of identification and estimation of pollution sources and for three separate benchmark data sets from UCI repository. The results show the proposed computational approach gives better performance than a human specialist, while offering many advantages over similar approaches found in the literature.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Bernardos PG, Vosniakos GC (2004) Optimizing feedforward artificial neural network architecture. Eng Appl Artif Intell 20(3):365–382CrossRef Bernardos PG, Vosniakos GC (2004) Optimizing feedforward artificial neural network architecture. Eng Appl Artif Intell 20(3):365–382CrossRef
2.
Zurück zum Zitat Yao X, Liu Y (1997) A new evolutionary system for evolving artificial neural networks. IEEE Trans Neural Netw 8(3):694–713MathSciNetCrossRef Yao X, Liu Y (1997) A new evolutionary system for evolving artificial neural networks. IEEE Trans Neural Netw 8(3):694–713MathSciNetCrossRef
3.
Zurück zum Zitat Rumelhart DE, Hinton GE, Willians RJ (1986) Learning internal representations of back-propagating error. Nat Biotechnol 323:533–536 Rumelhart DE, Hinton GE, Willians RJ (1986) Learning internal representations of back-propagating error. Nat Biotechnol 323:533–536
4.
Zurück zum Zitat Rojas R (1996) Neural networks, a systematic introduction, neural networks, a systematic introduction. Springer-Verlag, New York Rojas R (1996) Neural networks, a systematic introduction, neural networks, a systematic introduction. Springer-Verlag, New York
5.
Zurück zum Zitat Haykin S (1995) Neural networks. A comprehensive foundation, neural networks. A Comprehensive Foundation, Macmillan, New York Haykin S (1995) Neural networks. A comprehensive foundation, neural networks. A Comprehensive Foundation, Macmillan, New York
6.
Zurück zum Zitat Goldberg DE, Holland JH (1988) Genetic algorithms and machine learning: introduction to the special issue on genetic algorithms. Mach Learn 3:95–99 Goldberg DE, Holland JH (1988) Genetic algorithms and machine learning: introduction to the special issue on genetic algorithms. Mach Learn 3:95–99
7.
Zurück zum Zitat Farmer JD, Toffoli T, Wolfram S (1983) Cellular automata. Proceedings of an interdisciplinary workshop at Los Alamos, New Mexico, March 7–11 Farmer JD, Toffoli T, Wolfram S (1983) Cellular automata. Proceedings of an interdisciplinary workshop at Los Alamos, New Mexico, March 7–11
8.
Zurück zum Zitat Iyoda EM (2000) Inteligência computacional no projeto automático de redes neurais híbridas e redes neurofuzzy heterogêneas, Tese de Doutorado—Universidade Estadual de Campinas (UNICAMP)—Campinas, SP: [s.n.] Iyoda EM (2000) Inteligência computacional no projeto automático de redes neurais híbridas e redes neurofuzzy heterogêneas, Tese de Doutorado—Universidade Estadual de Campinas (UNICAMP)—Campinas, SP: [s.n.]
9.
Zurück zum Zitat Miller GF, Todd PM, Hedge SU (1991) Designing neural networks. Neural Netw 4:53–60CrossRef Miller GF, Todd PM, Hedge SU (1991) Designing neural networks. Neural Netw 4:53–60CrossRef
10.
Zurück zum Zitat Arifovic J (2001) Using genetic algorithms to select architecture of a feedforward artificial neural network. Physica A 289:574–594 Arifovic J (2001) Using genetic algorithms to select architecture of a feedforward artificial neural network. Physica A 289:574–594
11.
Zurück zum Zitat Whitley D, Starkweather T, Bogart C (1989) Genetic algorithm and neural networks: optimizing connections and connectivity. Computing 14:347–361 Whitley D, Starkweather T, Bogart C (1989) Genetic algorithm and neural networks: optimizing connections and connectivity. Computing 14:347–361
12.
Zurück zum Zitat Schafer JD, Caruana RA, Eshelman LJ (1990) Using genetic search to exploit the emergent behavior of neural networks. Physica D 42:244–248CrossRef Schafer JD, Caruana RA, Eshelman LJ (1990) Using genetic search to exploit the emergent behavior of neural networks. Physica D 42:244–248CrossRef
13.
Zurück zum Zitat Menczer F, Parisi D (1992) Evidence of hyperplanes in the genetic learning of neural networks. Biol Cybernet 66:283–289CrossRef Menczer F, Parisi D (1992) Evidence of hyperplanes in the genetic learning of neural networks. Biol Cybernet 66:283–289CrossRef
14.
Zurück zum Zitat Kitano H (1994) Evolution, complexity, entropy and artificial reality. Physica D 75:239–263CrossRef Kitano H (1994) Evolution, complexity, entropy and artificial reality. Physica D 75:239–263CrossRef
15.
Zurück zum Zitat Kitano H (1990) Designing neural networks using genetic algorithms with graph generation system. Complex Syst 4:461–476 Kitano H (1990) Designing neural networks using genetic algorithms with graph generation system. Complex Syst 4:461–476
16.
Zurück zum Zitat Harp S, Samad A, Guha A (1989) Toward the genetic synthesis of neural networks. In: Schafer JD (ed) Proceedings of the third international conference on genetic algorithms. Morgan Kaufman, San Mateo, CA, pp 762–767 Harp S, Samad A, Guha A (1989) Toward the genetic synthesis of neural networks. In: Schafer JD (ed) Proceedings of the third international conference on genetic algorithms. Morgan Kaufman, San Mateo, CA, pp 762–767
17.
Zurück zum Zitat Chen Z, Xiao J, Cheng J (1997) A program for automatic structure search. Proceedings of the 1997 international conference on neural networks, pp 308–311 Chen Z, Xiao J, Cheng J (1997) A program for automatic structure search. Proceedings of the 1997 international conference on neural networks, pp 308–311
18.
Zurück zum Zitat Koehn P (1994) Combining genetic algorithms and neural networks: the enconding problem, master thesis, University of Tennessee, Knoxville, EUA Koehn P (1994) Combining genetic algorithms and neural networks: the enconding problem, master thesis, University of Tennessee, Knoxville, EUA
19.
Zurück zum Zitat Bak P, Sneppen K (1993) Punctuated equilibrium and criticality in a simple model of evolution. Phys Rev Lett 71(24):4083–4086CrossRef Bak P, Sneppen K (1993) Punctuated equilibrium and criticality in a simple model of evolution. Phys Rev Lett 71(24):4083–4086CrossRef
20.
Zurück zum Zitat Sousa FL, Vlassov V, Ramos FM (2003) Generalized extremal optimization for solving complex optimal design problems. Lect Notes Comput Sci 2723:375–376 Sousa FL, Vlassov V, Ramos FM (2003) Generalized extremal optimization for solving complex optimal design problems. Lect Notes Comput Sci 2723:375–376
21.
Zurück zum Zitat Bak P (1996) How nature works. Copernicus, Springer-Verlag, New York Bak P (1996) How nature works. Copernicus, Springer-Verlag, New York
22.
Zurück zum Zitat Hansen P, Mladenovic N (1995) A variable neighborhood algorithm: a new metaheuristic for combinatorial optimization, abstracts of papers at optimization days, 112 Hansen P, Mladenovic N (1995) A variable neighborhood algorithm: a new metaheuristic for combinatorial optimization, abstracts of papers at optimization days, 112
23.
Zurück zum Zitat Hansen P, Mladenovic N (2003) Variable neighborhood search. In: Glover F, Kochenberger G (eds) Handbook of metaheuristics. Kluwer, Dordrecht, pp 145–184 Hansen P, Mladenovic N (2003) Variable neighborhood search. In: Glover F, Kochenberger G (eds) Handbook of metaheuristics. Kluwer, Dordrecht, pp 145–184
24.
Zurück zum Zitat Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Sci Agric 220(4598):671–680MathSciNet Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Sci Agric 220(4598):671–680MathSciNet
25.
Zurück zum Zitat Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, Reading Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, Reading
26.
Zurück zum Zitat Holland JH (1975) Adaptation in natural and artificial systems. University of Michigan Press, Englewood Cliffs Holland JH (1975) Adaptation in natural and artificial systems. University of Michigan Press, Englewood Cliffs
27.
Zurück zum Zitat Hornik K (1993) Some new results on neural network approximation. Neural Netw 6:1069–1072 Hornik K (1993) Some new results on neural network approximation. Neural Netw 6:1069–1072
28.
Zurück zum Zitat Bishop C (1995) Neural networks for pattern recognition. Oxford University Press, Oxford Bishop C (1995) Neural networks for pattern recognition. Oxford University Press, Oxford
29.
Zurück zum Zitat Ripley BD (1996) Pattern recognition and neural networks, pattern recognition and neural networks. Cambridge University Press, Cambridge Ripley BD (1996) Pattern recognition and neural networks, pattern recognition and neural networks. Cambridge University Press, Cambridge
30.
Zurück zum Zitat Roberti DR, Anfossi D, Campos Velho HF, Degrazia GA (2005) Estimating emission rate and pollutant source location. Cienc Nat (special volume):131–134 Roberti DR, Anfossi D, Campos Velho HF, Degrazia GA (2005) Estimating emission rate and pollutant source location. Cienc Nat (special volume):131–134
31.
Zurück zum Zitat Ferrero E, Anfossi D (1998) Comparison of PDFs, closures schemes and turbulence parameterizations in lagrangian stochastic models. Int J Environ Pollut 9:384–410 Ferrero E, Anfossi D (1998) Comparison of PDFs, closures schemes and turbulence parameterizations in lagrangian stochastic models. Int J Environ Pollut 9:384–410
32.
Zurück zum Zitat Ferrero E, Anfossi D, Brusasca G, Tinarelli G (1995) Lagrangian particle model LAMBDA: evaluation against tracer data. Int J Environ Pollut 5:360–374 Ferrero E, Anfossi D, Brusasca G, Tinarelli G (1995) Lagrangian particle model LAMBDA: evaluation against tracer data. Int J Environ Pollut 5:360–374
33.
Zurück zum Zitat Luz EFP, Campos Velho HF, Becceneri JC, Roberti DR (2007) Estimating atmospheric area source strength through particle swarm optimization, inverse problems, desing and optimization symposium IPDO-2007, April 16–18, Miami (FL), USA, 1:354–359 Luz EFP, Campos Velho HF, Becceneri JC, Roberti DR (2007) Estimating atmospheric area source strength through particle swarm optimization, inverse problems, desing and optimization symposium IPDO-2007, April 16–18, Miami (FL), USA, 1:354–359
34.
Zurück zum Zitat Paes FF, Campos Velho HF, Carvalho AR, Luz EP (2008) Identifing atmospheric pollutant sources using artificial neural networks, AGU 2008 joint assembly, Fort Lauderdale (FL), USA, pp 131–134 Paes FF, Campos Velho HF, Carvalho AR, Luz EP (2008) Identifing atmospheric pollutant sources using artificial neural networks, AGU 2008 joint assembly, Fort Lauderdale (FL), USA, pp 131–134
Metadaten
Titel
Metaheuristics for the feedforward artificial neural network (ANN) architecture optimization problem
verfasst von
Adenilson R. Carvalho
Fernando M. Ramos
Antonio A. Chaves
Publikationsdatum
01.11.2011
Verlag
Springer-Verlag
Erschienen in
Neural Computing and Applications / Ausgabe 8/2011
Print ISSN: 0941-0643
Elektronische ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-010-0504-3

Weitere Artikel der Ausgabe 8/2011

Neural Computing and Applications 8/2011 Zur Ausgabe