Top

Evolutionary Intelligence

Published in:

01-03-2008 | Review Article

Neuroevolution: from architectures to learning

Authors: Dario Floreano, Peter Dürr, Claudio Mattiussi

Published in: Evolutionary Intelligence | Issue 1/2008

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Artificial neural networks (ANNs) are applied to many real-world problems, ranging from pattern classification to robot control. In order to design a neural network for a particular task, the choice of an architecture (including the choice of a neuron model), and the choice of a learning algorithm have to be addressed. Evolutionary search methods can provide an automatic solution to these problems. New insights in both neuroscience and evolutionary biology have led to the development of increasingly powerful neuroevolution techniques over the last decade. This paper gives an overview of the most prominent methods for evolving ANNs with a special focus on recent advances in the synthesis of learning architectures.

previous article Genetic fuzzy systems: taxonomy, current research trends and prospects

next article Learning classifier systems: then and now

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Analog networks are collections of dynamical devices interconnected by links of varying strength. For example, genetic regulatory networks, metabolic networks, neural networks, or electronic circuits can be seen as analog networks.

Algorithms which combine evolutionary search with some kinds of local search are sometimes called memetic algorithms [53].

The two spaces are correlated if genotypes which are close in the evolutionary space correspond to phenotypes which are also close in the phenotype space.

An alternative approach to this are neural learning classifier systems. For example, Hurst and Bull [35] addressed the control of a simulated robot in a maze task. They used a population of neural networks acting as ‘rules’ controlling the robot. As evolution favored rules that led to succesful behavior, the set of rules adapted to the requirements of the task.

Ackley DH, Littman ML (1992) Interactions between learning and evolution. In: Langton C, Farmer J, Rasmussen S, Taylor C (eds) Artificial Life II: Proceedings volume of Santa Fe conference, vol XI. Addison Wesley, Redwood City, pp 487–510

Bailey CH, Giustetto M, Huang Y.-Y, Hawkins RD, Kandel ER (2000) Is heterosynaptic modulation essential for stabilizing Hebbian plasticity and memory? Nat Rev Neurosci 1(1):11–20CrossRef

Baldwin JM (1896) A new factor in evolution. Am Nat 30:441–451CrossRef

Banzhaf W, Nordin P, Keller RE, Francone FD (1998) Genetic programming—an introduction. In: On the automatic evolution of computer programs and its applications. Morgan Kaufmann, San Francisco

Barto AG (1995) Adaptive critic and the basal ganglia. In: Houk JC, Davis JL, Beiser DG (eds) Models of information processing in the basal ganglia. MIT Press, Cambridge, pp 215–232

Baxter J (1992) The evolution of learning algorithms for artificial neural networks. In: Green D, Bossomaier T (eds) Complex Systems. IOS Press

Beer RD, Gallagher JC (1992) Evolving dynamical neural networks for adaptive behavior. Adapt Behav 1:91–122CrossRef

Belew RK, McInerney J, Schraudolph NN (1992) Evolving networks: using the genetic algorithm with connectionistic learning. In: Langton CG, Taylor C, Farmer JD, Rasmussen S (eds) Proceedings of the 2nd Conference on Artificial Life. Addison-Wesley, Reading, pp 511–548

Blynel J, Floreano D (2003) Exploring the T-maze: evolving learning-like robot behaviors using CTRNNs. In: Raidl Ge AE (ed) 2nd European workshop on evolutionary robotics (EvoRob’2003)

10.

Bongard J (2002) Evolving modular genetic regulatory networks. In: Proceedings of the 2002 congress on evolutionary computation 2002, CEC ’02, vol 2, pp 1872–1877

11.

Chalmers DJ (1990) The evolution of learning: an experiment in genetic connectionism. In: Touretzky DS, Elman JL, Sejnowski T, Hinton GE (eds) Proceedings of the 1990 connectionist models summer school. Morgan Kaufmann, San Mateo, pp 81–90

12.

Chandra A, Yao X (2006) Ensemble learning using multi-objective evolutionary algorithms. J Math Model Algorithms 5(4):417–445MATHCrossRefMathSciNet

13.

Chellapilla K, Fogel D (2001) Evolving an expert checkers playing program without using humanexpertise. IEEE Trans Evol Comput 5(4):422–428CrossRef

14.

Dasdan A, Oflazer K (1993) Genetic synthesis of unsupervised learning algorithms. In: Proceedings of the 2nd Turkish symposium on artificial intelligence and ANNs. Department of Computer Engineering and Information Science, Bilkent University, Ankara

15.

DiPaolo E (2003) Evolving spike-timing-dependent plasticity for single-trial learning in robots. Phil Trans R Soc Lond A 361:2299–2319CrossRefMathSciNet

16.

Dürr P, Mattiussi C, Floreano D (2006) Neuroevolution with Analog Genetic Encoding. In: Parallel problem solving from nature—PPSN iX, vol 9. Springer, Berlin, pp 671–680

17.

Federici D (2005) Evolving developing spiking neural networks. In: Proceedings of CEC 2005 IEEE congress on evolutionary computation

18.

Fellous J-M, Linster C (1998) Computational models of neuromodulation. Neural Comput 10(4):771–805CrossRef

19.

Floreano D, Mattiussi C (2001) Evolution of spiking neural controllers for autonomous vision-based robots. In: Gomi T (ed) Evolutionary robotics. From intelligent robotics to artificial life. Springer, Tokyo

20.

Floreano D, Mondada F (1996) Evolution of plastic neurocontrollers for situated agents. In: Maes P, Matarić M, Meyer J, Pollack J, Roitblat H, Wilson S (eds) From animals to animats IV: proceedings of the 4th international conference on simulation of adaptive behavior. MIT Press-Bradford Books, Cambridge, pp 402–410

21.

Floreano D, Urzelai J (2000) Evolutionary robots with online self-organization and behavioral fitness. Neural Netw 13:431–443CrossRef

22.

Floreano D, Urzelai J (2001) Evolution of plastic control networks. Autonom Robots 11(3):311–317MATHCrossRef

23.

Fontanari JF, Meir R (1991) Evolving a learning algorithm for the binary perceptron. Network 2:353–359CrossRef

24.

Funahashi K, Nakamura Y (1993) Approximation of dynamical systems by continuous time recurrent neural networks. Neural Netw 6(6):801–806CrossRef

25.

Geard NL, Wiles J (2003) Structure and dynamics of a gene network model incorporating small RNAs. In: Proceedings of 2003 congress on evolutionary computation, pp 199–206

26.

Gerstner W (1999) Spiking neurons. In: Maass W, Bishop CM (eds) Pulsed neural networks. MIT Press-Bradford Books, Cambridge

27.

Gomez F, Miikkulainen R (1997) Incremental evolution of complex general behavior. Adapt Behav 5(3–4):317–342CrossRef

28.

Gruau F (1995) Automatic definition of modular neural networks. Adapt Behav 3(2):151–183CrossRef

29.

Gruau, F, Whitley, D, and Pyeatt, L (1996) A comparison between cellular encoding and direct encoding for genetic neural networks. In: Koza JR, Goldberg DE, Fogel DB, Riolo RL (eds) Genetic programming 1996: proceedings of the first annual conference. MIT Press, Stanford University, pp 81–89

30.

Hansen N, Ostermeier A (2001) Completely derandomized self-adaptation in evolution strategies. Evol Comput 9(2):159–195CrossRef

31.

Haykin, S (1999) Neural networks. a comprehensive foundation, 2nd edn. Prentice Hall, Upper Saddle RiverMATH

32.

Hebb DO (1949) The organisation of behavior. Wiley, New York

33.

Hinton GE, Nowlan SJ (1987) How learning can guide evolution. Complex Syst 1:495–502MATH

34.

Hodgkin AL, Huxley AF (1952) A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol (Lond) 108:500–544

35.

Hurst J, Bull L (2006) A neural learning classifier system with self-adaptive constructivism for mobile robot control. Artif Life 12 (3):353–380CrossRef

36.

Husbands P, Harvey I, Cliff D, Miller G (1994) The use of genetic algorithms for the development of sensorimotor control systems. In: Gaussier P, Nicoud J-D (eds) From perceptin to action. IEEE Press, Los Alamitos

37.

Husbands P, Smith T, Jakobi N, O’Shea M (1998) Better living through chemistry: evolving gasnets for robot control. Connect Sci 10:185–210CrossRef

38.

Igel, C (2003) Neuroevolution for reinforcement learning using evolution strategies. In: Sarker R, et al (eds) Congress on evolutionary computation, vol 4. IEEE Press, New York, pp 2588–2595

39.

Katz PS (1999) What are we talking about? Modes of neuronal communication. In: Katz P (eds) Beyond neurotransmission: neuromodulation and its importance for information processing, chap 1. Oxford University Press, Oxford, pp 1–28

40.

Kitano H (1990) Designing neural networks by genetic algorithms using graph generation system. Complex Syst J 4:461–476MATH

41.

Korkin M, Nawa NE, de Garis H (1998) A ’spike interval information coding’ representation for ATR’s CAM-brain machine (CBM) In: Proceedings of the 2nd international conference on evolvable systems: from biology to hardware (ICES’98). Springer, Heidelberg

42.

Koza JR (1994) Genetic programming II: automatic discovery of reusable programs. MIT Press, CambridgeMATH

43.

Magg S, Philippides A (2006) Gasnets and CTRNNs : a comparison in terms of evolvability. In: From animals to animats 9: proceedings of the 9th international conference on simulation of adaptive behavior. Springer, Heidelberg, pp 461–472

44.

Mattiussi C, Floreano D (2004) Evolution of analog networks using local string alignment on highly reorganizable genomes. In: Zebulum RS et al (eds) NASA/DoD conference on evolvable hardware (EH’2004), pp 30–37

45.

Mattiussi C, Dürr P, Floreano D (2007a) Center of mass encoding: a self-adaptive representation with adjustable redundancy for real-valued parameters. In: GECCO 2007. ACM Press, New York, pp 1304–1311CrossRef

46.

Mattiussi C, Marbach D, Dürr P, Floreano D (2007b) The age of analog networks. AI Magazine (in press)

47.

Mayley G (1996) Landscapes, learning costs and genetic assimilation. Evol Comput 4(3):213–234

48.

McHale G, Husbands P (2004) Gasnets and other evolvable neural networks applied to bipedal locomotion. In: Schaal S (ed) Proceedings from animals to animats 8: proceedings of the 8th international conference on simulation of adaptive behaviour (SAB’2004). MIT Press, Cambridge, pp 163–172

49.

Mizutani E, Dreyfus SE (1998) Totally model-free reinforcement learning by actor-critic elman networks in non-markovian domains. In: Proceedings of the IEEE world congress on computational intelligence. IEEE Press, New York

50.

Montague P, Dayan P, Sejnowski T (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16(5):1936–1947

51.

Montana D, Davis L (1989) Training feed forward neural networks using genetic algorithms. In: Proceedings of the 11th international joint conference on artificial intelligence. Morgan Kaufmann, San Mateo, pp 529–538

52.

Moriarty DE, Miikkulainen R (1996) Efficient reinforcement learning through symbiotic evolution. Machine Learn 22:11–32

53.

Moscato P (1989) On evolution, search, optimization, genetic algorithms and martial arts: towards memetic algorithms. In: Technical report C3P 826, Pasadena

54.

Niv Y, Joel D, Meilijson I, Ruppin E (2002) Evolution of reinforcement learning in uncertain environments: A simple explanation for complex foraging behaviors. Adapt Behav 10(1):5–24CrossRef

55.

Nolfi S, Floreano D (1999) Learning and evolution. Auton Robots 7(1):89–113CrossRef

56.

Nolfi S, Parisi D (1996) Learning to adapt to changing environments in evolving neural networks. Adapt Behav 5(1):75–98CrossRef

57.

Nolfi S, Miglino O, Parisi D (1994) Phenotypic plasticity in evolving neural networks. In: Gaussier P, Nicoud J-D (eds) From perception to action. IEEE Press, Los Alamitos

58.

Pfeifer R, Scheier C (1999) Understanding Intelligence. MIT Press, Cambridge

59.

Purves D (1994) Neural activity in the growth of the brain. Cambridge University Press, Cambridge

60.

Quartz S, Sejnowski TJ (1997) The neural basis of cognitive development: a constructivist manifesto. Behav Brain Sci 4:537–555CrossRef

61.

Radcliffe NJ (1991) Form an analysis and random respectful recombination. In: Belew RK, Booker LB (eds) Proceedings of the 4th international conference on genetic algorithms. Morgan Kaufmann, San Mateo

62.

Rechenberg I (1973) Evolutionsstrategie—Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. Fommann-Holzboog, Stuttgart

63.

Reil T (1999) Dynamics of gene expression in an artificial genome—implications for biological and artificial ontogeny. In: Proceedings of the 5th European conference on artificial life, pp 457–466

64.

Reil T (2003) On growth, form and computers. In: Artificial genomes as models of gene regulation. Academic Press, London, pp 256–277

65.

Reisinger J, Miikkulainen R (2007) Acquiring evolvability through adaptive representations. In: Proceedings of genetic and evolutionary computation conference (GECCO 2007)

66.

Reisinger J, Bahçeci E, Karpov I, Miikkulainen R (2007) Coevolving strategies for general game playing. In: Proceedings of the IEEE symposium on computational intelligence and games (CIG-2007)

67.

Rieke F, Warland D, van Steveninck R, Bialek W (1997) Spikes. Exploring the neural code. MIT Press, Cambridge

68.

Rumelhart DE, Hinton GE, Williams RJ (1986a) Learning representations by back-propagation of errors. Nature 323:533–536CrossRef

69.

Rumelhart DE, McClelland J, the PDP Research Group (1986b) Parallel distributed processing: explorations in the microstructure of cognition. Foundations, vol 1. MIT Press-Bradford Books, Cambridge

70.

Saggie K, Keinan A, Ruppin E (2004) Spikes that count: rethinking spikiness in neurally embedded systems. Neurocomputing 58-60:303–311CrossRef

71.

Sasaki T, Tokoro M (1997) Adaptation toward changing environments: Why Darwinian in nature?. In: Husbands P, Harvey I (eds) Proceedings of the 4th European conference on artificial life. MIT Press, Cambridge

72.

Schaffer JD, Whitley D, Eshelman LJ (1992) Combinations of genetic algorithms and neural networks: a survey of the state of the art. In: Whitley D, Schaffer JD (eds) Proceedings of an international workshop on the combinations of genetic algorithms and neural networks (COGANN-92). IEEE Press, New York

73.

Schraudolph NN, Belew RK (1992) Dynamic parameter encoding for genetic algorithms. Machine Learn 9:9–21

74.

Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275(5306):1593–1599CrossRef

75.

Shapiro J (2005) A 21st century view of evolution: genome system architecture, repetitive DNA, and natural genetic engineering. Gene 345(1):91–100CrossRef

76.

Siddiqi A, Lucas S (1998) A comparison of matrix rewriting versus direct encoding for evolving neural networks. In: Proceedings of the 1998 IEEE international conference on evolutionary computation. Piscataway, NJ, pp 392–397

77.

Singer W, Gray CM (1995) Visual feature integration and the temporal correlation hypothesis. Annu Rev Neurosci 18:555–586CrossRef

78.

Soltoggio A, Duerr P, Mattiussi C, Floreano D (2007) Evolving neuromodulatory topologies for reinforcement learning-like problems. In: Angeline P, Michaelewicz M, Schonauer G, Yao X, Zalzala Z (eds) Proceedings of the 2007 congress on evolutionary computation. IEEE Press, New York

79.

Stanley K, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evol Comput 10(2):99–127CrossRef

80.

Stanley KO, Miikkulainen R (2004) Competitive coevolution through evolutionary complexification. J Artif Intell Res 21:63–100

81.

Stanley K, Kohl N, Sherony R, Miikkulainen R (2005a) Neuroevolution of an automobile crash warning system. In: Proceedings of genetic and evolutionary computation conference (GECCO 2005)

82.

Stanley KO, Cornelius R, Miikkulainen R, D’Silva T, Gold A (2005b) Real-time learning in the nero video game. In: Proceedings of the artificial intelligence and interactive digital entertainment conference (AIIDE 2005) demo papers

83.

Sutton RS (1988) Learning to predict by the method of temporal difference. Machine Learn 3:9–44

84.

Sutton RS, Barto AG (1998) Reinforcement learning. an introduction. MIT Press, Cambridge

85.

Trianni V, Ampatzis C, Christensen A, Tuci E, Dorigo M, Nolfi S (2007) From solitary to collective behaviours: decision making and cooperation. In: Advances in artificial life, proceedings of ECAL 2007. Lecture Notes in Artificial Intelligence, vol LNAI 4648. Springer, Berlin, pp 575–584

86.

Tuci E, Quinn M, Harvey I (2002) An evolutionary ecological approach to the study of learning behavior using a robot-based model. Adapt Behav 10(3–4):201–221CrossRef

87.

Urzelai J, Floreano D (2001) Evolution of adaptive synapses: robots with fast adaptive behavior in new environments. Evol Comput 9:495–524CrossRef

88.

Whitley D, Starkweather T, Bogart C (1990) Genetic algorithms and neural networks: optimizing connections and connectivity. Parallel Comput 14:347–361CrossRef

89.

Widrow B, Hoff ME (1960) Adaptive switching circuits. In: Proceedings of the 1960 IRE WESCON convention, vol IV, New York. IRE. Reprinted in Anderson and Rosenfeld, 1988, pp 96–104

90.

Yamauchi BM, Beer RD (1994) Sequential behavior and learning in evolved dynamical neural networks. Adapt Behav 2(3):219–246CrossRef

91.

Yao X (1999) Evolving artificial neural networks. Proc IEEE 87(9):1423–1447CrossRef

Title: Neuroevolution: from architectures to learning
Authors: Dario Floreano
Peter Dürr
Claudio Mattiussi
Publication date: 01-03-2008
Publisher: Springer-Verlag
Published in: Evolutionary Intelligence / Issue 1/2008
Print ISSN: 1864-5909
Electronic ISSN: 1864-5917
DOI: https://doi.org/10.1007/s12065-007-0002-4

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 1/2008

Foreword

Genetic fuzzy systems: taxonomy, current research trends and prospects

Learning classifier systems: then and now

An interdisciplinary perspective on artificial immune systems

Dedication: Dr. Lawrence J. Fogel (1928–2007)

Premium Partner