Skip to main content
Top
Published in: Neural Computing and Applications 9/2017

08-02-2016 | Original Article

Rapid application prototyping for hardware modular spiking neural network architectures

Authors: Sandeep Pande, Fearghal Morgan, Finn Krewer, Jim Harkin, Liam McDaid, Brian McGinley

Published in: Neural Computing and Applications | Issue 9/2017

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Spiking neural networks (SNNs) are well suited for functions such as data/pattern classification, estimation, prediction, signal processing and robotic control applications. Whereas the real-world embedded applications are often multi-functional with orthogonal or contradicting functional requirements. The EMBRACE hardware modular SNN architecture has been previously reported as an embedded computing platform for complex real-world applications. The EMBRACE architecture employs genetic algorithm (GA) for training the SNN which offers faster prototyping of SNN applications, but exhibits a number of limitations including poor scalability and search space explosions for the evolution of large-scale, complex, real-world applications. This paper investigates the limitations of evolving real-world embedded applications with orthogonal functional goals on hardware SNN using GA-based training. This paper presents a novel, fast and efficient application prototyping technique using the EMBRACE hardware modular SNN architecture and the GA-based evolution platform. Modular design and evolution of a robotic navigational controller application decomposed into obstacle avoidance controller and speed and direction manager application subtasks is presented. The proposed modular evolution technique successfully integrates the orthogonal functionalities of the application and helps to overcome contradicting application scenarios gracefully. Results illustrate that the modular evolution of the application reduces the SNN configuration search space and complexity for the GA-based SNN evolution, offering rapid and successful prototyping of complex applications on the hardware SNN platform. The paper presents validation results of the evolved robotic application implemented on the EMBRACE architecture prototyped on Xilinx Virtex-6 FPGA interacting with the player-stage robotics simulator.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Footnotes
1
EMulating Biologically-inspiRed ArChitectures in hardwarE.
 
2
The task decomposition for the robotic navigational controller modular application design presented in this paper has been done manually based on input vector partitioning and analysing the orthogonality of the functional requirements of the application.
 
3
The markers for the proposed robotic navigational controller application are chosen by the designer such that the robot can progress towards the destination by following the markers in sequence. The markers can also be chosen by applying meta-heuristic algorithms to the classical Travelling Salesman Problem which is currently out of scope of this paper.
 
Literature
1.
go back to reference Haykin S (1998) Neural networks: a comprehensive foundation, 2nd edn. Prentice Hall PTR, Upper Saddle RiverMATH Haykin S (1998) Neural networks: a comprehensive foundation, 2nd edn. Prentice Hall PTR, Upper Saddle RiverMATH
2.
go back to reference Maass W (1997) Networks of spiking neurons: the third generation of neural network models. Neural Netw 10(9):1659–1671CrossRef Maass W (1997) Networks of spiking neurons: the third generation of neural network models. Neural Netw 10(9):1659–1671CrossRef
3.
go back to reference Gerstner W, Kistler WM (2002) Spiking neuron models: single neurons, populations, plasticity. Cambridge University Press, CambridgeCrossRefMATH Gerstner W, Kistler WM (2002) Spiking neuron models: single neurons, populations, plasticity. Cambridge University Press, CambridgeCrossRefMATH
4.
go back to reference Bohte S, Kok JN (2005) Applications of spiking neural networks. Inf Process Lett 95(6):519–520CrossRefMATH Bohte S, Kok JN (2005) Applications of spiking neural networks. Inf Process Lett 95(6):519–520CrossRefMATH
5.
go back to reference Pearson M, Pipe A, Mitchinson B, Gurney K, Melhuish C, Gilhespy I, Nibouche M (2007) Implementing spiking neural networks for real-time signal-processing and control applications: a model-validated FPGA approach. IEEE Trans Neural Netw 18(5):1472–1487CrossRef Pearson M, Pipe A, Mitchinson B, Gurney K, Melhuish C, Gilhespy I, Nibouche M (2007) Implementing spiking neural networks for real-time signal-processing and control applications: a model-validated FPGA approach. IEEE Trans Neural Netw 18(5):1472–1487CrossRef
6.
go back to reference Harkin J, Morgan F, McDaid L, Hall S, McGinley B, Cawley S (2009) A reconfigurable and biologically inspired paradigm for computation using network-on-chip and spiking neural networks. Int J Reconfig Comput 2009:2:1–2:13CrossRef Harkin J, Morgan F, McDaid L, Hall S, McGinley B, Cawley S (2009) A reconfigurable and biologically inspired paradigm for computation using network-on-chip and spiking neural networks. Int J Reconfig Comput 2009:2:1–2:13CrossRef
7.
go back to reference Cawley S, Morgan F, McGinley B, Pande S, McDaid L, Carrillo S, Harkin J (2011) Hardware spiking neural network prototyping and application. Genet Progr Evolvable Mach 12:257–280CrossRef Cawley S, Morgan F, McGinley B, Pande S, McDaid L, Carrillo S, Harkin J (2011) Hardware spiking neural network prototyping and application. Genet Progr Evolvable Mach 12:257–280CrossRef
8.
go back to reference Pande S, Morgan F, Cawley S, Bruintjes T, Smit G, McGinley B, Carrillo S, Harkin J, McDaid L (2013). Modular neural tile architecture for compact embedded hardware spiking neural network. Neural Process Lett 38(2):131–153CrossRef Pande S, Morgan F, Cawley S, Bruintjes T, Smit G, McGinley B, Carrillo S, Harkin J, McDaid L (2013). Modular neural tile architecture for compact embedded hardware spiking neural network. Neural Process Lett 38(2):131–153CrossRef
9.
go back to reference Pande S, Morgan F, Smit G, Bruintjes T, Rutgers J, McGinley B, Cawley S, Harkin J, McDaid L (2013) Fixed latency on-chip interconnect for hardware spiking neural network architectures. Parallel Comput 39:357–371CrossRef Pande S, Morgan F, Smit G, Bruintjes T, Rutgers J, McGinley B, Cawley S, Harkin J, McDaid L (2013) Fixed latency on-chip interconnect for hardware spiking neural network architectures. Parallel Comput 39:357–371CrossRef
10.
go back to reference Pande S, Morgan F, Cawley S, McGinley B, Harkin J, Carrillo S, McDaid L (2011) Addressing the hardware resource requirements of network-on-chip based neural architectures. In: International conference on neural computation theory and applications (NCTA), Paris, France Pande S, Morgan F, Cawley S, McGinley B, Harkin J, Carrillo S, McDaid L (2011) Addressing the hardware resource requirements of network-on-chip based neural architectures. In: International conference on neural computation theory and applications (NCTA), Paris, France
11.
go back to reference Morgan F, Cawley S, McGinley B, Pande S, McDaid LJ, Glackin B, Maher J, Harkin J (2009) Exploring the evolution of NoC-based spiking neural networks on FPGAs. In: International conference on field-programmable technology, 2009 (FPT 2009). IEEE, pp 300–303 Morgan F, Cawley S, McGinley B, Pande S, McDaid LJ, Glackin B, Maher J, Harkin J (2009) Exploring the evolution of NoC-based spiking neural networks on FPGAs. In: International conference on field-programmable technology, 2009 (FPT 2009). IEEE, pp 300–303
12.
go back to reference Brooks R (1986) A robust layered control system for a mobile robot. IEEE J Robot Autom 2(1):14–23CrossRef Brooks R (1986) A robust layered control system for a mobile robot. IEEE J Robot Autom 2(1):14–23CrossRef
13.
go back to reference Vaughan R (2008) Massively multi-robot simulation in stage. Swarm Intell 2:189–208CrossRef Vaughan R (2008) Massively multi-robot simulation in stage. Swarm Intell 2:189–208CrossRef
14.
go back to reference Johannes S, Wieringa BM, Matzke M, Mnte TF (1996) Hierarchical visual stimuli: electrophysiological evidence for separate left hemispheric global and local processing mechanisms in humans. Neurosci Lett 210(2):111–114CrossRef Johannes S, Wieringa BM, Matzke M, Mnte TF (1996) Hierarchical visual stimuli: electrophysiological evidence for separate left hemispheric global and local processing mechanisms in humans. Neurosci Lett 210(2):111–114CrossRef
15.
go back to reference Van Essen D, Anderson C, Felleman D (1992) Information processing in the primate visual system: an integrated systems perspective. Science 255(5043):419–423CrossRef Van Essen D, Anderson C, Felleman D (1992) Information processing in the primate visual system: an integrated systems perspective. Science 255(5043):419–423CrossRef
16.
go back to reference Binzegger T, Douglas R, Martin K (2007) Stereotypical bouton clustering of individual neurons in cat primary visual cortex. J Neurosci 27(45):12242–12254CrossRef Binzegger T, Douglas R, Martin K (2007) Stereotypical bouton clustering of individual neurons in cat primary visual cortex. J Neurosci 27(45):12242–12254CrossRef
17.
go back to reference Happel B, Murre J (1994) Design and evolution of modular neural network architectures. Neural Netw 7(6):985–1004CrossRef Happel B, Murre J (1994) Design and evolution of modular neural network architectures. Neural Netw 7(6):985–1004CrossRef
18.
go back to reference Auda G, Kamel M (1999) Modular neural networks: a survey. Int J Neural Syst 9(02):129–151CrossRef Auda G, Kamel M (1999) Modular neural networks: a survey. Int J Neural Syst 9(02):129–151CrossRef
19.
go back to reference Ronco E, Gawthrop P (1995) Modular neural networks: a state of the art. Rapport Technique CSC95026, Center of System and Control University of Glasgow 1:1–22 Ronco E, Gawthrop P (1995) Modular neural networks: a state of the art. Rapport Technique CSC95026, Center of System and Control University of Glasgow 1:1–22
20.
go back to reference Osherson D, Weinstein S, Stob M (1993) Modular learning. MIT Press, CambridgeMATH Osherson D, Weinstein S, Stob M (1993) Modular learning. MIT Press, CambridgeMATH
21.
go back to reference Guan S, Li S, Tan SK (2004) Neural network task decomposition based on output partitioning. J Inst Eng Singap 44:78–89 Guan S, Li S, Tan SK (2004) Neural network task decomposition based on output partitioning. J Inst Eng Singap 44:78–89
22.
go back to reference Lu B-L, Ito M (1999) Task decomposition and module combination based on class relations: a modular neural network for pattern classification. IEEE Trans Neural Netw 10(5):1244–1256CrossRef Lu B-L, Ito M (1999) Task decomposition and module combination based on class relations: a modular neural network for pattern classification. IEEE Trans Neural Netw 10(5):1244–1256CrossRef
23.
go back to reference Thangavelautham J, Deleuterio G (2004) A neuroevolutionary approach to emergent task decomposition. In: Proceedings of 8th parallel problem solving from nature. Springer, Berlin, pp 991–1000 Thangavelautham J, Deleuterio G (2004) A neuroevolutionary approach to emergent task decomposition. In: Proceedings of 8th parallel problem solving from nature. Springer, Berlin, pp 991–1000
24.
go back to reference Khare V, Yao X, Sendhoff B, Jin Y, Wersing H (2005) Co-evolutionary modular neural networks for automatic problem decomposition. In: The 2005 IEEE congress on evolutionary computation, 2005, vol 3. IEEE, pp 2691–2698 Khare V, Yao X, Sendhoff B, Jin Y, Wersing H (2005) Co-evolutionary modular neural networks for automatic problem decomposition. In: The 2005 IEEE congress on evolutionary computation, 2005, vol 3. IEEE, pp 2691–2698
25.
go back to reference Santos J, Alexandre L, de Sá JM (2006) Modular neural network task decomposition via entropic clustering. In: Sixth international conference on intelligent systems design and applications, 2006. ISDA ’06, vol 1. IEEE, pp 62–67 Santos J, Alexandre L, de Sá JM (2006) Modular neural network task decomposition via entropic clustering. In: Sixth international conference on intelligent systems design and applications, 2006. ISDA ’06, vol 1. IEEE, pp 62–67
26.
go back to reference Hebb DO (2005) The organization of behavior: a neuropsychological theory. Psychology Press, Hove Hebb DO (2005) The organization of behavior: a neuropsychological theory. Psychology Press, Hove
27.
go back to reference Hinton G, Sejnowski T (1999) Unsupervised learning: foundations of neural computation. The MIT press, Cambridge Hinton G, Sejnowski T (1999) Unsupervised learning: foundations of neural computation. The MIT press, Cambridge
28.
go back to reference Natschläger T, Ruf B (1998) Online clustering with spiking neurons using temporal coding. Prog Neural Process 10:33–42CrossRefMATH Natschläger T, Ruf B (1998) Online clustering with spiking neurons using temporal coding. Prog Neural Process 10:33–42CrossRefMATH
29.
go back to reference Bohte S, La Poutré H, Kok J (2002) Unsupervised clustering with spiking neurons by sparse temporal coding and multilayer RBF networks. IEEE Trans Neural Netw 13(2):426–435CrossRef Bohte S, La Poutré H, Kok J (2002) Unsupervised clustering with spiking neurons by sparse temporal coding and multilayer RBF networks. IEEE Trans Neural Netw 13(2):426–435CrossRef
30.
go back to reference Landis F, Ott T, Stoop R (2010) Hebbian self-organizing integrate-and-fire networks for data clustering. Neural Comput 22(1):273–288MathSciNetCrossRefMATH Landis F, Ott T, Stoop R (2010) Hebbian self-organizing integrate-and-fire networks for data clustering. Neural Comput 22(1):273–288MathSciNetCrossRefMATH
31.
go back to reference Hopfield JJ (1995) Pattern recognition computation using action potential timing for stimulus representation. Nature 376(6535):33–36CrossRef Hopfield JJ (1995) Pattern recognition computation using action potential timing for stimulus representation. Nature 376(6535):33–36CrossRef
32.
go back to reference Gerstner W, van Hemmen JL (1992) Associative memory in a network of spiking neurons. Netw Comput Neural Syst 3(2):139–164CrossRefMATH Gerstner W, van Hemmen JL (1992) Associative memory in a network of spiking neurons. Netw Comput Neural Syst 3(2):139–164CrossRefMATH
33.
go back to reference Zamani M, Sadeghian A, Chartier S (2010) A bidirectional associative memory based on cortical spiking neurons using temporal coding. In: IJCNN, pp 1–8 Zamani M, Sadeghian A, Chartier S (2010) A bidirectional associative memory based on cortical spiking neurons using temporal coding. In: IJCNN, pp 1–8
34.
go back to reference Alpaydin E (2010) Introduction to machine learning. The MIT Press, CambridgeMATH Alpaydin E (2010) Introduction to machine learning. The MIT Press, CambridgeMATH
35.
go back to reference Bohte S, Kok J, La Poutre H (2002) Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing 48(1):17–37CrossRefMATH Bohte S, Kok J, La Poutre H (2002) Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing 48(1):17–37CrossRefMATH
36.
go back to reference Legenstein R, Naeger C, Maass W (2005) What can a neuron learn with spike-timing-dependent plasticity? Neural Comput 17(11):2337–2382MathSciNetCrossRefMATH Legenstein R, Naeger C, Maass W (2005) What can a neuron learn with spike-timing-dependent plasticity? Neural Comput 17(11):2337–2382MathSciNetCrossRefMATH
37.
go back to reference Legenstein R, Pecevski D, Maass W (2008) A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback. PLoS Comput Biol 4(10):e1000180MathSciNetCrossRef Legenstein R, Pecevski D, Maass W (2008) A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback. PLoS Comput Biol 4(10):e1000180MathSciNetCrossRef
38.
39.
go back to reference Farries M, Fairhall A (2007) Reinforcement learning with modulated spike timing-dependent synaptic plasticity. J Neurophysiol 98(6):3648–3665CrossRef Farries M, Fairhall A (2007) Reinforcement learning with modulated spike timing-dependent synaptic plasticity. J Neurophysiol 98(6):3648–3665CrossRef
40.
go back to reference Florian R (2005) A reinforcement learning algorithm for spiking neural networks. In: Seventh international symposium on symbolic and numeric algorithms for scientific computing, 2005 (SYNASC 2005). IEEE Florian R (2005) A reinforcement learning algorithm for spiking neural networks. In: Seventh international symposium on symbolic and numeric algorithms for scientific computing, 2005 (SYNASC 2005). IEEE
41.
go back to reference Florian R (2007) Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Comput 19(6):1468–1502MathSciNetCrossRefMATH Florian R (2007) Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Comput 19(6):1468–1502MathSciNetCrossRefMATH
42.
go back to reference Izhikevich E (2007) Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb Cortex 17(10):2443–2452CrossRef Izhikevich E (2007) Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb Cortex 17(10):2443–2452CrossRef
43.
go back to reference Vasilaki E, Frémaux N, Urbanczik R, Senn W, Gerstner W (2009) Spike-based reinforcement learning in continuous state and action space: when policy gradient methods fail. PLoS Comput Biol 5(12):e1000586CrossRef Vasilaki E, Frémaux N, Urbanczik R, Senn W, Gerstner W (2009) Spike-based reinforcement learning in continuous state and action space: when policy gradient methods fail. PLoS Comput Biol 5(12):e1000586CrossRef
44.
go back to reference Di Paolo E (2002) Spike-timing dependent plasticity for evolved robots. Adapt Behav 10(3–4):243–263CrossRef Di Paolo E (2002) Spike-timing dependent plasticity for evolved robots. Adapt Behav 10(3–4):243–263CrossRef
45.
go back to reference Hagras H, Pounds-Cornish A, Colley M, Callaghan V, Clarke G (2004) Evolving spiking neural network controllers for autonomous robots. In: 2004 IEEE international conference on robotics and automation, 2004. Proceedings. ICRA ’04 vol 5. IEEE, pp 4620–4626 Hagras H, Pounds-Cornish A, Colley M, Callaghan V, Clarke G (2004) Evolving spiking neural network controllers for autonomous robots. In: 2004 IEEE international conference on robotics and automation, 2004. Proceedings. ICRA ’04 vol 5. IEEE, pp 4620–4626
46.
go back to reference Mitchell M (1998) An introduction to genetic algorithms. MIT Press, CambridgeMATH Mitchell M (1998) An introduction to genetic algorithms. MIT Press, CambridgeMATH
47.
go back to reference Vainbrand D, Ginosar R (2011) Scalable network-on-chip architecture for configurable neural networks. Microprocess Microsyst 35(2):152–166CrossRef Vainbrand D, Ginosar R (2011) Scalable network-on-chip architecture for configurable neural networks. Microprocess Microsyst 35(2):152–166CrossRef
Metadata
Title
Rapid application prototyping for hardware modular spiking neural network architectures
Authors
Sandeep Pande
Fearghal Morgan
Finn Krewer
Jim Harkin
Liam McDaid
Brian McGinley
Publication date
08-02-2016
Publisher
Springer London
Published in
Neural Computing and Applications / Issue 9/2017
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-015-2136-0

Other articles of this Issue 9/2017

Neural Computing and Applications 9/2017 Go to the issue

Premium Partner