Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben
Neural Processing Letters
Erschienen in: Neural Processing Letters 2/2020

13.08.2020

Spiking Neural Networks: Background, Recent Development and the NeuCube Architecture

verfasst von: Clarence Tan, Marko Šarlija, Nikola Kasabov

Erschienen in: Neural Processing Letters | Ausgabe 2/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper reviews recent developments in the still-off-the-mainstream information and data processing area of spiking neural networks (SNN)—the third generation of artificial neural networks. We provide background information about the functioning of biological neurons, discussing the most important and commonly used mathematical neural models. Most relevant information processing techniques, learning algorithms, and applications of spiking neurons are described and discussed, focusing on feasibility and biological plausibility of the methods. Specifically, we describe in detail the functioning and organization of the latest version of a 3D spatio-temporal SNN-based data machine framework called NeuCube, as well as it’s SNN-related submodules. All described submodules are accompanied with formal algorithmic formulations. The architecture is highly relevant for the analysis and interpretation of various types of spatio-temporal brain data (STBD), like EEG, NIRS, fMRI, but we highlight some of the recent both STBD- and non-STBD-based applications. Finally, we summarise and discuss some open research problems that can be addressed in the future. These include, but are not limited to: application in the area of EEG-based BCI through transfer learning; application in the area of affective computing through the extension of the NeuCube framework which would allow for a biologically plausible SNN-based integration of central and peripheral nervous system measures. Matlab implementation of the NeuCube’s SNN-related module is available for research and teaching purposes.
Vorheriger Artikel Label Embedding for Multi-label Classification Via Dependence Maximization

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Anhänge
Nur mit Berechtigung zugänglich
Fußnoten
Literatur
1.
Hubel DH, Wiesel TN (1959) Receptive fields of single neurones in the cat’s striate cortex. J Physiol 148(3):574–591
2.
Fukushima K (1979) Neural network model for a mechanism of pattern recognition unaffected by shift in position-neocognitron. IEICE Tech Rep A 62(10):658–665
3.
Rosenblatt F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 65(6):386
4.
Hannun A, Case C, Casper J, Catanzaro B, Diamos G, Elsen E, Prenger R, Satheesh S, Sengupta S, Coates A et al (2014) Deep speech: scaling up end-to-end speech recognition. arXiv preprint arXiv:​1412.​5567
5.
Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252MathSciNet
6.
Ouyang W, Wang X (2013) Joint deep learning for pedestrian detection. In: Proceedings of the IEEE international conference on computer vision, pp 2056–2063
7.
Cireşan D, Meier U, Schmidhuber J (2017) Multi-column deep neural networks for image classification. arXiv preprint arXiv:​1202.​2745
8.
Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M et al (2016) Mastering the game of go with deep neural networks and tree search. Nature 529(7587):484
9.
Šarlija M, Jurišić F, Popović S (2017) A convolutional neural network based approach to QRS detection. In: 10th international symposium on image and signal processing and analysis (ISPA). IEEE, pp 121–125
10.
Ganapathy N, Swaminathan R, Deserno TM (2018) Deep learning on 1-d biosignals: a taxonomy-based survey. Yearb Med Inform 27(01):098–109
11.
Drubach D (2000) The brain explained. Prentice Hall Health, Upper Saddle River
12.
Bengio Y, Lee D-H, Bornschein J, Mesnard T, Lin Z (2015) Towards biologically plausible deep learning. arXiv preprint arXiv:​1502.​04156
13.
Maass W (1997) Networks of spiking neurons: the third generation of neural network models. Neural Netw 10(9):1659–1671
14.
Trentin E, Schwenker F, El Gayar N, Abbas HM (2018) Off the mainstream: advances in neural networks and machine learning for pattern recognition. Neural Process Lett 48(2):643–648
15.
Kasabov NK (2014) Neucube: a spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data. Neural Netw 52:62–76
16.
Herz AV, Gollisch T, Machens CK, Jaeger D (2006) Modeling single-neuron dynamics and computations: a balance of detail and abstraction. Science 314(5796):80–85MathSciNetMATH
17.
Lapicque L (1907) Recherches quantitatives sur l’excitation electrique des nerfs traitee comme une polarization. J Physiol Pathol Gen 9:620–635
18.
Abbott LF (1999) Lapicque’s introduction of the integrate-and-fire model neuron (1907). Brain Res Bull 50(5–6):303–304
19.
Gerstner W, Kistler WM (2002) Spiking neuron models: single neurons, populations, plasticity. Cambridge University Press, CambridgeMATH
20.
Izhikevich EM (2003) Simple model of spiking neurons. IEEE Trans Neural Netw 14(6):1569–1572MathSciNet
21.
Izhikevich EM (2004) Which model to use for cortical spiking neurons? IEEE Trans Neural Netw 15(5):1063–1070
22.
Hodgkin AL, Huxley AF (1952) A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol 117(4):500–544
23.
Wilson C, Callaway J (2000) Coupled oscillator model of the dopaminergic neuron of the substantia nigra. J Neurophysiol 83(5):3084–3100
24.
FitzHugh R (1961) Fitzhugh-nagumo simplified cardiac action potential model. Biophys J 1:445–466
25.
Hindmarsh JL, Rose R (1984) A model of neuronal bursting using three coupled first order differential equations. Proc R Soc Lond B 221(1222):87–102
26.
Morris C, Lecar H (1981) Voltage oscillations in the barnacle giant muscle fiber. Biophys J 35(1):193–213
27.
Katsumata S, Sakai K, Toujoh S, Miyamoto A, Nakai J, Tsukada M, Kojima H (2008) Analysis of synaptic transmission and its plasticity by glutamate receptor channel kinetics models and 2-photon laser photolysis. In: Proceedings of ICONIP
28.
Huguenard JR (2000) Reliability of axonal propagation: the spike doesn’t stop here. Proc Nat Acad Sci 97(17):9349–9350
29.
Kasabov N (2010) To spike or not to spike: a probabilistic spiking neuron model. Neural Netw 23(1):16–19
30.
Sengupta N, Kasabov N (2017) Spike-time encoding as a data compression technique for pattern recognition of temporal data. Inf Sci 406:133–145
31.
Adrian ED (1926) The impulses produced by sensory nerve endings. J Physiol 61(1):49–72
32.
Gautrais J, Thorpe S (1998) Rate coding versus temporal order coding: a theoretical approach. Biosystems 48(1–3):57–65
33.
Lestienne R (2001) Spike timing, synchronization and information processing on the sensory side of the central nervous system. Prog Neurobiol 65(6):545–591
34.
Bohte SM (2004) The evidence for neural information processing with precise spike-times: a survey. Nat Comput 3(2):195–206MathSciNetMATH
35.
Thorpe SJ (1990) Spike arrival times: a highly efficient coding scheme for neural networks. In: Eckmiller R, Hartmann G, Hauske G (eds) Parallel processing in neural systems and computers. North-Holland Elsevier, pp 91–94
36.
Brette R (2015) Philosophy of the spike: rate-based vs. spike-based theories of the brain. Front Syst Neurosci 9:151
37.
Mohemmed A, Schliebs S, Matsuda S, Kasabov N (2011) Method for training a spiking neuron to associate input-output spike trains. In: Engineering applications of neural networks. Springer, pp 219–228
38.
Thorpe S, Gautrais J (1998) Rank order coding. In: Computational neuroscience. Springer, pp 113–118
39.
Buzsaki G (2006) Rhythms of the brain. Oxford University Press, OxfordMATH
40.
Petro B, Kasabov N, Kiss RM (2019) Selection and optimization of temporal spike encoding methods for spiking neural networks. IEEE Trans Neural Netw Learn Syst 31(2):358–370
41.
Kasabov NK (2018) Time-space. Spiking neural networks and brain-inspired artificial intelligence. Springer, Berlin
42.
Rueckauer B, Lungu I-A, Hu Y, Pfeiffer M (2016) Theory and tools for the conversion of analog to spiking convolutional neural networks. arXiv preprint arXiv:​1612.​04052
43.
Diehl PU, Cook M (2015) Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Front Comput Neurosci 9:99
44.
Diehl PU, Neil D, Binas J, Cook M, Liu S-C, Pfeiffer M (2015) Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In: 2015 international joint conference on neural networks (IJCNN). IEEE, pp 1–8
45.
Cao Y, Chen Y, Khosla D (2015) Spiking deep convolutional neural networks for energy-efficient object recognition. Int J Comput Vis 113(1):54–66MathSciNet
46.
Merolla P, Arthur J, Akopyan F, Imam N, Manohar R, Modha DS (2011) A digital neurosynaptic core using embedded crossbar memory with 45pj per spike in 45 nm. In: Custom integrated circuits conference (CICC), 2011 IEEE. IEEE, pp 1–4
47.
O’Connor P, Neil D, Liu S-C, Delbruck T, Pfeiffer M (2013) Real-time classification and sensor fusion with a spiking deep belief network. Front. Neurosci. 7:178
48.
49.
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
50.
Esser SK, Appuswamy R, Merolla P, Arthur JV, Modha DS (2015) Backpropagation for energy-efficient neuromorphic computing. In: Advances in neural information processing systems, pp 1117–1125
51.
Merolla PA, Arthur JV, Alvarez-Icaza R, Cassidy AS, Sawada J, Akopyan F, Jackson BL, Imam N, Guo C, Nakamura Y et al (2014) A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345(6197):668–673
52.
Gerstner W, Kempter R, van Hemmen JL, Wagner H (1996) A neuronal learning rule for sub-millisecond temporal coding. Nature 383(6595):76
53.
Bi G-Q, Poo M-M (1998) Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J Neurosci 18(24):10464–10472
54.
Gerstner W, Ritz R, Van Hemmen JL (1993) Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns. Biol Cybern 69(5–6):503–515MATH
55.
Cassenaer S, Laurent G (2007) Hebbian STDP in mushroom bodies facilitates the synchronous flow of olfactory information in locusts. Nature 448(7154):709
56.
Jacob V, Brasier DJ, Erchova I, Feldman D, Shulz DE (2007) Spike timing-dependent synaptic depression in the in vivo barrel cortex of the rat. J Neurosci 27(6):1271–1284
57.
Mu Y, Poo M-M (2006) Spike timing-dependent ltp/ltd mediates visual experience-dependent plasticity in a developing retinotectal system. Neuron 50(1):115–125
58.
Song S, Miller KD, Abbott LF (2000) Competitive hebbian learning through spike-timing-dependent synaptic plasticity. Nat Neurosci 3(9):919
59.
Kheradpisheh SR, Ganjtabesh M, Thorpe SJ, Masquelier T (2018) Stdp-based spiking deep convolutional neural networks for object recognition. Neural Netw 99:56–67
60.
Tavanaei A, Maida AS (2017) A spiking network that learns to extract spike signatures from speech signals. Neurocomputing 240:191–199
61.
Hirsch H-G, Pearce D (2000) The aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions. In: ASR2000-automatic speech recognition: challenges for the new Millenium ISCA Tutorial and Research Workshop (ITRW)
62.
Kasabov N et al (1998) Evolving fuzzy neural networks-algorithms, applications and biological motivation. Methodologies for the conception, design and application of soft computing. World Sci 1:271–274
63.
Kasabov NK (2007) Evolving connectionist systems: the knowledge engineering approach. Springer, BerlinMATH
64.
Wysoski SG, Benuskova L, Kasabov N (2010) Evolving spiking neural networks for audiovisual information processing. Neural Netw 23(7):819–835
65.
Kasabov N, Dhoble K, Nuntalid N, Indiveri G (2013) Dynamic evolving spiking neural networks for on-line spatio-and spectro-temporal pattern recognition. Neural Netw 41:188–201
66.
Kasabov N (2012) Neucube evospike architecture for spatio-temporal modelling and pattern recognition of brain signals. In: IAPR workshop on artificial neural networks in pattern recognition. Springer, pp 225–243
67.
Lichtsteiner P, Delbruck T (2005) A 64 \(\times \) 64 AER logarithmic temporal derivative silicon retina. In: Research in microelectronics and electronics PhD, vol 2. IEEE, pp 202–205
68.
Nuntalid N, Dhoble K, Kasabov N (2011) EEG classification with BSA spike encoding algorithm and evolving probabilistic spiking neural network. In: International conference on neural information processing. Springer, pp 451–460
69.
Kasabov N, Scott NM, Tu E, Marks S, Sengupta N, Capecci E, Othman M, Doborjeh MG, Murli N, Hartono R et al (2016) Evolving spatio-temporal data machines based on the neucube neuromorphic framework: design methodology and selected applications. Neural Netw 78:1–14MATH
70.
Talairach J, Tournoux P (1988) Co-planar stereotaxic atlas of the human brain: 3-dimensional proportional system: an approach to cerebral imaging
71.
Evans AC, Collins DL, Mills S, Brown E, Kelly R, Peters TM (1993) 3D statistical neuroanatomical models from 305 MRI volumes. In: Nuclear science symposium and medical imaging conference. 1993 IEEE conference record. IEEE, pp 1813–1817
72.
Kasabov NK, Doborjeh MG, Doborjeh ZG (2016) Mapping, learning, visualization, classification, and understanding of fmri data in the neucube evolving spatiotemporal data machine of spiking neural networks. IEEE Trans Neural Netw Learn Syst 28(4):887–899
73.
Kasabov N, Zhou L, Doborjeh MG, Doborjeh ZG, Yang J (2016) New algorithms for encoding, learning and classification of fmri data in a spiking neural network architecture: a case on modeling and understanding of dynamic cognitive processes. IEEE Trans Cogn Dev Syst 9(4):293–303
74.
Abbott A, Sengupta N, Kasabov N (2016) Which method to use for optimal structure and function representation of large spiking neural networks: a case study on the neucube architecture. In: International joint conference on neural networks (IJCNN), 2016 . IEEE, pp 1367–1372
75.
Taylor D, Scott N, Kasabov N, Capecci E, Tu E, Saywell N, Chen Y, Hu J, Hou Z-G (2014) Feasibility of neucube SNN architecture for detecting motor execution and motor intention for use in BCI applications. In: International joint conference on neural networks (IJCNN), 2014 . IEEE, pp 3221–3225
76.
Hu J, Hou Z-G, Chen Y-X, Kasabov N, Scott N (2014) EEG-based classification of upper-limb ADL using SNN for active robotic rehabilitation. In: 2014 5th IEEE RAS & EMBS international conference on biomedical robotics and biomechatronics. IEEE, pp 409–414
77.
Othman M, Kasabov N, Tu E, Feigin V, Krishnamurthi R, Hou Z, Chen Y, Hu J (2014) Improved predictive personalized modelling with the use of spiking neural network system and a case study on stroke occurrences data. In: 2014 international joint conference on neural networks (IJCNN). IEEE, pp 3197–3204
78.
Doborjeh ZG, Kasabov N, Doborjeh MG, Sumich A (2018) Modelling peri-perceptual brain processes in a deep learning spiking neural network architecture. Sci Rep 8(1):8912
79.
Paulun L, Wendt A, Kasabov NK (2018) A retinotopic spiking neural network system for accurate recognition of moving objects using neucube and dynamic vision sensors. Front Comput Neurosci 12:42
80.
Sengupta N, McNabb CB, Kasabov N, Russell BR (2018) Integrating space, time, and orientation in spiking neural networks: a case study on multimodal brain data modeling. IEEE Trans Neural Netw Learn Syst 99:1–15MathSciNet
81.
Oreilly C, Gosselin N, Carrier J, Nielsen T (2014) Montreal archive of sleep studies: an open-access resource for instrument benchmarking and exploratory research. J Sleep Res 23(6):628–635
82.
Koelstra S, Muhl C, Soleymani M, Lee J-S, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I (2012) Deap: A database for emotion analysis; using physiological signals. IEEE Trans Affect Comput 3(1):18–31
83.
Soleymani M, Lichtenauer J, Pun T, Pantic M (2012) A multimodal database for affect recognition and implicit tagging. IEEE Trans Affect Comput 3(1):42–55
Metadaten
Titel
Spiking Neural Networks: Background, Recent Development and the NeuCube Architecture
verfasst von
Clarence Tan
Marko Šarlija
Nikola Kasabov
Publikationsdatum
13.08.2020
Verlag
Springer US
Erschienen in
Neural Processing Letters / Ausgabe 2/2020
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-020-10322-8

Weitere Artikel der Ausgabe 2/2020

Neural Processing Letters 2/2020 Zur Ausgabe

OriginalPaper

State Estimation of Static Neural Networks with Mixed Delay

OriginalPaper

Synchronization of Stochastic Complex Dynamical Networks with Mixed Time-Varying Coupling Delays

OriginalPaper

Feature Selection Method Based on Differential Correlation Information Entropy

OriginalPaper

Approximately Optimal Control of Discrete-Time Nonlinear Switched Systems Using Globalized Dual Heuristic Programming

OriginalPaper

A Review of Dynamic Maps for 3D Human Motion Recognition Using ConvNets and Its Improvement

OriginalPaper

A New Fixed-Time Stability Criterion and Its Application to Synchronization Control of Memristor-Based Fuzzy Inertial Neural Networks with Proportional Delay

Neuer Inhalt

    Bildnachweise