Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Neural Processing Letters
Published in: Neural Processing Letters 1/2016

01-08-2016

A Cognitive Architecture Based on a Learning Classifier System with Spiking Classifiers

Authors: David Howard, Larry Bull, Pier-Luca Lanzi

Published in: Neural Processing Letters | Issue 1/2016

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Learning classifier systems (LCS) are population-based reinforcement learners that were originally designed to model various cognitive phenomena. This paper presents an explicitly cognitive LCS by using spiking neural networks as classifiers, providing each classifier with a measure of temporal dynamism. We employ a constructivist model of growth of both neurons and synaptic connections, which permits a genetic algorithm to automatically evolve sufficiently-complex neural structures. The spiking classifiers are coupled with a temporally-sensitive reinforcement learning algorithm, which allows the system to perform temporal state decomposition by appropriately rewarding “macro-actions”, created by chaining together multiple atomic actions. The combination of temporal reinforcement learning and neural information processing is shown to outperform benchmark neural classifier systems, and successfully solve a robotic navigation task.
previous article Computation by Time
next article A Network of Neural Oscillators for Fractal Pattern Recognition

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
Literature
1.
2.
Bonarini A (1998) Reinforcement distribution to fuzzy classifiers. In: Proceedings of the IEEE world congress on computational intelligence (WCCI)—evolutionary computation, IEEE Computer Press, pp 51–56
3.
Boyan JA, Moore AW (1995) Generalization in reinforcement learning: safely approximating the value function. In: Tesauro G, Touretzky DS, Leen TK (eds) Advances in neural information processing systems 7. The MIT Press, Cambridge, pp 369–376
4.
Bull L (2002) On using constructivism in neural classifier systems. In: Merelo J, Adamidis P, Beyer HG, Fernandez-Villacanas JL, Schwefel HP (eds) Parallel problem solving from nature—PPSN VII. Springer, New York, pp 558–567CrossRef
5.
Bull L, Hurst J (2003) A neural learning classifier system with self-adaptive constructivism. In: Proceedings of the IEEE congress on evolutionary computation, IEEE Press, pp 991–997
6.
Butz MV, Herbort O (2008) Context-dependent predictions and cognitive arm control with XCSF. In: Ryan C, Keijzer M (eds) Proceedings of the genetic and evolutionary computation conference, GECCO 2008, ACM, Atlanta, 12–16 July 2008, pp 1357–1364
7.
Butz MV, Lanzi PL, Wilson SW (2006) Hyper-ellipsoidal conditions in xcs: rotation, linear approximation, and solution structure. In: Proceedings of the 8th annual conference on genetic and evolutionary computation (GECCO ’06), ACM Press, New York, pp 1457–1464. doi:10.​1145/​1143997.​1144237
8.
9.
Cazangi RR, Zuben FJV, Figueiredo M (2003) A classifier system in real applications for robot navigation. In: Proceedings of the IEEE congress on evolutionary computation, IEEE, pp 574–580
10.
Churchill AW, Fernando C (2014) An evolutionary cognitive architecture made of a bag of networks. Evol Intell 7(3):169–182CrossRef
11.
Donnart JY, Meyer JA (1996) Learning reactive and planning rules in a motivationally autonomous animat. IEEE Trans Syst Man Cybern 26(3):381–395CrossRef
12.
Dorigo M, Colombetti M (1994) Robot shaping: developing autonomous agents through learning. Artif Intell 71(2):321–370CrossRef
13.
14.
15.
16.
Floreano D, Mattiussi C (2001) Evolution of spiking neural controllers for autonomous vision-based robots. Lect Notes Comput Sci 2217:38–61CrossRefMATH
17.
Floreano D, Schoeni N, Caprari G, Blynel J (2002) Evolutionary bitsnspikes. In: Proceedings of the eight international conference on artificial life, MIT Press
18.
Gerstner W, Kistler W (2002) Spiking neuron models—single neurons, populations, plasticity. Cambridge University Press, CambridgeCrossRefMATH
19.
Hagras H, Sobh T (2002) Intelligent learning and control of autonomous robotic agents operating in unstructured environments. Inf Sci 145(1):1–12MathSciNetCrossRefMATH
20.
He P, Jagannathan S (2007) Reinforcement learning neural-network-based controller for nonlinear discrete-time systems with input constraints. In: Proceedings of the IEEE transactions on systems, man, and cybernetics, Part B: cybernetics, 37(2):425–436
21.
Hodgkin AL, Huxley AF (1952) A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol 117(4):500CrossRef
22.
Holland JH (1975) Adaptation in natural and artificial systems. The University of Michigan Press, Ann Arbor
23.
Holland JH (1976) Adaptation. In: Rosen R, Snell F (eds) Progress in theoretical biology. Academic Press, New York
24.
Holland JH, Reitman JS (1978) Cognitive systems based on adaptive algorithms. In: Waterman DA, Hayes-Roth F (eds) Pattern-directed inference systems. Academic Press, Orlando, pp 313–329
25.
Howard G, Bull L, Lanzi PL (2010) A spiking neural representation for xcsf. In: Proceedings of the IEEE congress on evolutionary computation (CEC), IEEE, pp 1–8
26.
Howard GD, Bull L (2008) On the effects of node duplication and connection-oriented constructivism in neural XCSF. In: Ryan C, Keijzer M (eds) In: Proceedings of the genetic and evolutionary computation conference, GECCO 2008, Atlanta, Companion Material, ACM, 12–16 July 2008, pp 1977–1984
27.
Howard GD, Bull L, Lanzi PL (2009) Towards continuous actions in continuous space and time using self-adaptive constructivism in neural XCSF. In: Proceedings of the 11th annual conference on genetic and evolutionary computation, GECCO ’09, ACM, New York, pp 1219–1226. doi:10.​1145/​1569901.​1570065
28.
Hurst J, Bull L (2006) A neural learning classifier system with self-adaptive constructivism for mobile robot control. Artif Life 12(3):353–380CrossRef
29.
Hurst J, Bull L, Melhuish C (2002) TCS learning classifier system controller on a real robot. Lect Notes Comput Sci 2439:588–600CrossRef
30.
31.
32.
Lanzi PL, Loiacono D, Wilson SW, Goldberg DE (2005) XCS with computed prediction in continuous multistep environments. In: Proceedings of the IEEE congress on evolutionary computation, IEEE, pp 2032–2039
33.
Lanzi PL, Loiacono D, Wilson SW, Goldberg DE (2006) Classifier prediction based on tile coding. In: Proceedings of the 8th annual conference on genetic and evolutionary computation, GECCO ’06, ACM, New York, pp 1497–1504
34.
Maass W (1997) Networks of spiking neurons: the third generation of neural network models. Neural Netw 10(9):1659–1671CrossRef
35.
Michel O (2004) WebotsTM: professional mobile robot simulation. Int J Adv Robot Syst 1(1):39–42
36.
Moioli RC, Vargas PA, Zuben FJV (2007) Analysing learning classifier systems in reactive and non-reactive robotic tasks. In: Bacardit J, Bernadó-Mansilla E, Butz MV, Kovacs T, Llorà X, Takadama K (eds) International workshop on learning classifier systems IWLCS, lecture notes in computer science, Springer, New York, vol 4998, pp 286–305
37.
Pipe AG, Carse B (2002) First results from experiments in fuzzy classifier system architectures for mobile robotics. Lect Notes Comput Sci 2439:578–587CrossRef
38.
39.
Quartz SR, Sejnowski TJ (1997) The neural basis of cognitive development: a constructivist manifesto. Behav Brain Sci 20(04):537–556
40.
Rechenberg I (1973) Evolutionsstrategie: optimierung technischer systeme nach prinzipien der biologischen evolution. Frommann-Holzboog, Stuttgart
41.
Rumelhart D, McClelland J (1986) Parallel distributed processing, vol 1 & 2. MIT Press, Cambridge
42.
Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80(1):1–27
43.
Shouval H, Gavornik J (2011) A single spiking neuron that can represent interval timing: analysis, plasticity and multi-stability. J Comput Neurosci 30(2):489–499MathSciNetCrossRef
44.
Stolzmann W (1999) Latent learning in khepera robots with anticipatory classifier systems. In: Lanzi PL, Stolzmann W, Wilson SW (eds) 2nd international workshop on learning classifier systems. Orlando, pp 290–297
45.
Studley M, Bull L (2005) X-TCS: accuracy-based learning classifier system robotics. In: Proceedings of the IEEE congress on evolutionary computation, IEEE, pp 2099–2106
46.
Sutton RS (1996) Generalization in reinforcement learning: successful examples using sparse coarse coding. In: Touretzky DS, Mozer MC, Hasselmo ME (eds) Advances in neural information processing systems 8. MIT Press, Cambridge, pp 1038–1044
47.
Sutton RS, Precup D, Singh S (1999) Between mdps and semi-mdps: a framework for temporal abstraction in reinforcement learning. Artif Intell 112(1–2):181–211MathSciNetCrossRefMATH
48.
Watkins C (1989) Learning from delayed rewards. PhD thesis, Cambridge University, Psychology Department, Cambridge
49.
Webb A, Hart E, Ross P, Lawson A (2003) Controlling a simulated khepera with an XCS classifier system with memory. In: Banzhaf W, Christaller T, Dittrich P, Kim JT, Ziegler J (eds) Proceedings of the advances in artificial life, 7th European conference, ECAL 2003, lecture notes in computer science, vol 2801, Springer, Dortmund, pp 885–892, 14–17 Sept 2003
50.
Wilson SW (2000) Get real! xcs with continuous-valued inputs. In: Lanzi PL, Stolzmann W, Wilson SW (eds) Learning classifier systems, from foundations to applications, LNAI-1813. Springer, New York, pp 209–219
51.
Wilson SW (2001) Function approximation with a classifier system. In: Spector L, Goodman ED, Wu A, Langdon WB, Voigt HM, Gen M, Sen S, Dorigo M, Pezeshk S, Garzon MH, Burke E (eds) Proceedings of the genetic and evolutionary computation conference (GECCO-2001). Morgan Kaufmann, San Francisco, pp 974–981
52.
Wilson SW (2001b) Mining oblique data with XCS. In: Lanzi PL, Stolzmann W, Wilson SW (eds) Advances in learning classifier systems, third international workshop, IWLCS 2000, LNCS, vol 1996, Springer, Heidelberg, pp 158–176
Metadata
Title
A Cognitive Architecture Based on a Learning Classifier System with Spiking Classifiers
Authors
David Howard
Larry Bull
Pier-Luca Lanzi
Publication date
01-08-2016
Publisher
Springer US
Published in
Neural Processing Letters / Issue 1/2016
Print ISSN: 1370-4621
Electronic ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-015-9451-4

Other articles of this Issue 1/2016

Neural Processing Letters 1/2016 Go to the issue

OriginalPaper

Motor Imagery Electroencephalograph Classification Based on Optimized Support Vector Machine by Magnetic Bacteria Optimization Algorithm

OriginalPaper

Computation by Time

OriginalPaper

An Improved Neural Network with Random Weights Using Backtracking Search Algorithm

OriginalPaper

A Network of Neural Oscillators for Fractal Pattern Recognition

OriginalPaper

Unsupervised Clustering Based an Adaptive Particle Swarm Optimization Algorithm

OriginalPaper

A Resource Aware MapReduce Based Parallel SVM for Large Scale Image Classifications