Necessary conditions for STDP-based pattern recognition learning in a memristive spiking neural network
Introduction
The development of spiking neural networks (SNNs) has been mostly driven by the hope of replication the extraordinary low energy consumption and high computing efficiency of biological systems in a solution of so-called anthropogenic tasks (text and speech processing, pattern recognition, decision making, prediction, etc.) (Gerstner & Kistler, 2002). SNNs comprise layers (typically input, hidden and output) of artificial neurons (that collect, integrate and fire spikes) and connecting them synapses that represent strength (or weight) of the connection. Synaptic weights are tunable during training. Circuits based on complementary metal oxide semiconductor (CMOS) elements have been successfully used to implement SNNs, e.g. TrueNorth (Merolla et al., 2014), SpiNNaker (Furber, Galluppi, Temple, & Plana, 2014), Loihi (Davies et al., 2018), and others (Chen, Kumar, Sumbul, Knag, & Krishnamurthy, 2019). However, as the CMOS digital devices were not created or optimized for the purposes of neuromorphic (brain-inspired) computing, they do not faithfully emulate synapses and hence lack the intrinsic hardware training capability. In this respect memristors (“resistors with memory”) can emulate synapses (Strukov, Snider, Stewart, & Williams, 2008) and partially neurons (Pickett, Medeiros-Ribeiro, & Williams,2013) more faithfully because they share fundamentally similar operation mechanisms with their bio-counterparts: both mechanisms are closely associated with ion drift or diffusion and analog character of signal processing (Xia & Yang, 2019). Moreover, memristor-based neuromorphic networks with organization of the weights in a crossbar architecture have an ability to perform massively parallel and highly energy efficient vector-by-matrix multiplication directly at the site where data is stored, using the Kirchhoff electric current summation rules (Ielmini & Waser, 2016).
There are two most common types of memristor-based neural networks, namely deep neural networks (DNNs) and SNNs. Promising DNN schemes, such as multilayer perceptrons (Choi et al., 2017, Emelyanov et al., 2016, Li et al., 2018, Merrikh-Bayat et al., 2018, Mikhaylov et al., 2018, Silva et al., 2020), sparse coding networks (Cai et al., 2019), long short-term memory (Li et al., 2019), networks with reinforcement learning (Wang et al., 2019) and others (Akhmetov and James, 2019, Dowling et al., 2020, Sun et al., 2019) have been presented with the use of analog memristive weights between artificial software- or hardware-based neurons. However, training in these networks typically relies on a various kinds of error back-propagation algorithm, the efficient implementation of which is challenging due to the necessity of a global (operating at the level of a whole network) method for calculation of weight updates and the requirement for memristors with high cycle-to-cycle and device-to-device reproducibility (Del Valle, Ramírez, Rozenberg, & Schuller, 2018).
SNNs, in contrast to DNNs, encode information in rates and timing of spikes that gives more subtle dynamic responses and richer representations of an object features applied to the network input (Lobo, Del Ser, Bifet, & Kasabov, 2020). This is especially evident in the processing of temporal sequences such as speech and video, in the generation of control signals for mobile and robotic devices (Xia & Yang, 2019). Additionally, SNN training is based on local rules of synaptic weight update, requiring only information from pre- and post-synaptic neurons. This could be a significant advantage for compact and low power implementations of a real-time training, since the weight update does not require calculating each time the error gradients for neurons in the deeper layers of a network, and scaling to more complex networks (Merolla et al., 2014). Local rules potentially could provide correct modification of recurrent (backward and intra-layer) connections, including interneuron inhibition, or competition between neurons, which can significantly reduce requirements to the size of labeled training datasets (Demin & Nekhaev, 2018). However, despite their great potential, the computational abilities of SNNs have not been demonstrated as much as those of DNNs, mainly because of great but insufficient study of effective operating algorithms and learning rules, especially in hardware terms.
One of the most perspective local SNN learning rules is a Spike-Timing Dependent Plasticity (STDP), which is a type of Hebbian learning (Caporale & Dan, 2008) and can be successfully realized with different memristors (Ielmini and Waser, 2016, Jo et al., 2010, Kim et al., 2015, Kim et al., 2017, Minnekhanov et al., 2019, Prezioso et al., 2016). According to the basic pairwise model of STDP, a synaptic weight increases if the pre-synaptic neuron generates a spike just before the post-synaptic one (indicating a causal relationship), and a synaptic weight decreases if the post- synaptic neuron spikes just before the pre- synaptic one (reflecting a non-causal relationship) (Bi & Poo, 1998). Memristor implementation of STDP could also be modulated via heterosynaptic plasticity or dopamine-like assistance mechanisms which allows the realization of bio-inspired reinforcement learning algorithms (Maier et al., 2016, Nikiruy, Emelyanov et al., 2019). This makes the STDP mechanism the foundation both for supervised and unsupervised learning of memristor-based SNNs (Covi et al., 2016).
Although quite a lot number of works implementing STDP learning in memristor-based SNNs (Covi et al., 2018, Emelyanov et al., 2020, Nikiruy, Emelyanov, Rylkov, Sitnikov and Demin, 2019, Prezioso et al., 2018, Serb et al., 2016, Wang et al., 2018) and their simulation models (Ambrogio et al., 2016, Bill and Legenstein, 2014, Boyn et al., 2017, Lobov et al., 2020, Moraitis et al., 2017, Qu et al., 2020, Querlioz et al., 2013) have been demonstrated, it is not totally clear how STDP experimentally realized on memristors deals with training convergence issues. What parameters of a so-called “STDP window” (weight update dependence on inter-spike time intervals) and memristor characteristics are appropriate for STDP-based training of memristive networks to converge to an expected solution? One of the early thorough research in Querlioz et al. (2013) showed in a simulation that learning of a single-layer SNN with a quite simple exponential model of memristor weight update is largely robust with respect to the high enough variations of the minimum and maximum values of memristor conductivities, increments in the weight update, the limited resolution of the memristor conductance and the read disturb effect, the stimulus coding schemes, and the presence of some noise spikes at the inputs. At the same time, it is not yet clear how these remarkable results map to more complex forms of STDP and corresponding STDP window shapes observed in experiments with real memristors and hardware neurons, how the variations of experimentally observed STDP window parameters influence the direction of weight change under specific stimuli, and, at last, how these issues correlate with an accuracy of SNN solving some pattern recognition task. The present work addresses these issues from experimental, simulation and theoretical points of view that could be of high importance in practice when searching for memristor devices and pulse shapes appropriate for SNN learning convergence.
All physical experiments were done with (CoFeB)(LiNbO3) nanocomposite based memristors that demonstrate very appealing characteristics (Nikiruy, Emelyanov et al., 2019, Rylkov et al., 2018).
Section snippets
Analog neuron
To generate bi-triangular voltage pulses (spikes) with adjustable amplitude and duration parameters, a special leakyintegrate-and-fire (LIF) neuron scheme was developed. The main feature of this circuit differing it from many other hardware realizations of neurons (Acciarito et al., 2017, Ambrogio et al., 2016, Mahalanabis et al., 2016, Serrano-Gotarredona et al., 2013, Wu et al., 2015) is the generation of a bi-triangular spike and its transmission in both output and input directions of the
Memristive STDP
The NC memristors demonstrate resistive switching with the resistance ratio more than 103 and retention time more than 10s (, are the memristor resistances in OFF and ON resistive states, respectively). Typical I-V curves, endurance and resistive state time dependencies are presented in Supplementary Figs. S7, S8. The switching mechanism is based on the conductive filaments formation and destruction due to electromigration of oxygen vacancies controlled by the percolation
Spike-rate dependent plasticity
The simulation results show that the convergence of thememristor-based single-layer SNN training can be achieved in a wide, but not in the entire range of parameters of the STDP window model. It can be seen that the ratios of the amplitudes (), widths () and window branch relaxation times () are much more important than their absolute values. An influence of these ratios can most clearly be understood in a ‘post-pre-post’ spike triplet pairing scheme, when the two weight updates
Conclusions
In conclusion, we have experimentally found and theoretically proved the necessary conditions for guaranteeing the near-optimal clusterization learning in population-rate coding neural networks with either spike- or rate-based neurons. These conditions rely on the formulated ‘correlation growth-anticorrelation decay’ principle which provides the potentiation of weights of co-active input–output neuron pairs and weight decay between pre- and postsynaptic neurons that are in antiphase of their
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This work was supported by the Russian Foundation for Basic Research (Grant No. 18-29-23041) in part of analog neuron realization, SNN simulation, and building the theoretical model, and Russian Science Foundation (Grant No. 16-19-10233) in part of STDP implementation using (CoFeB)(LiNbO3) nanocomposite memristor. Measurements were carried out on the equipment of the Resource centers under support of the NRC “Kurchatov Institute” (#1055).
Authors are thankful to Prof. A.V. Sitnikov
References (73)
- et al.
Hardware design of LIF with Latency neuron model with memristive STDP synapses
IEEE Transactions on Very Large Scale Integration (VLSI) Systems
(2017) - et al.
Analogue signal and image processing with large memristor crossbars
Nature Electronics
(2018) - et al.
Spiking Neural Networks and online learning: An overview and perspectives
Neural Networks
(2020) - et al.
Perceptrons from memristors
Neural Networks
(2020) - et al.
A CMOS spiking neuron for Dense memristor-synapse connectivity for brain-inspired computing
- et al.
Probabilistic neural network with memristive crossbar circuits
- et al.
Unsupervised learning by spike timing dependent plasticity in phase change memory (PCM) synapses
Frontiers in Neuroscience
(2016) - et al.
Synaptic modifications in cultured hippocampal neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic cell type
The Journal of Neuroscience
(1998) - et al.
A compound memristive synapse model for statistical learning through STDP in spiking neural networks
Frontiers in Neuroscience
(2014) - et al.
Learning through ferroelectric domain dynamics in solid-state synapses
Neural Computation
(2017)
A fully integrated reprogrammable memristor–CMOS system for efficient multiply–accumulate operations
Nature Electronics
Spike timing–dependent Plasticity: A Hebbian learning rule
Annual Review of Neuroscience
A 4096-Neuron 1M-Synapse 3.8-pJ/SOP spiking neural network with On-Chip STDP Learning and Sparse Weights in 10-nm FinFET CMOS
IEEE Journal of Solid-State Circuits
A recipe for creating ideal hybrid memristive-CMOS neuromorphic processing systems
Applied Physics Letters
Experimental demonstration of feature extraction and dimensionality reduction using memristor networks
Nano Letters
Analog memristive synapse in spiking networks implementing unsupervised learning
Frontiers in Neuroscience
Spike-driven threshold-based learning with memristive synapses and neuromorphic silicon neurons
Journal of Physics D (Applied Physics)
Loihi: A neuromorphic Manycore processor with On-Chip Learning
IEEE Micro
Challenges in materials and devices for resistive-switching-based neuromorphic computing
Journal of Applied Physics
Recurrent spiking neural network learning based on a competitive maximization of neuronal activity
Frontiers in Neuroinformatics
Maximum likelihood from incomplete data via the EM algorithm
The Journal of the Royal Statistical Society, Series B
Probabilistic Memristive networks: Application of a Master Equation to Networks of binary ReRAM cells
First steps towards the realization of a double layer perceptron based on organic memristive devices
Advances
Self-adaptive STDP-based learning of a spiking neuron with nanocomposite memristive weights
Nanotechnology
The SpiNNaker project
Proceedings of the IEEE
Spiking neuron models
A triplet spike-timing-dependent plasticity model generalizes the Bienenstock-Cooper-Munro rule to higher-order spatiotemporal correlations
Proceedings of the National Academy of Sciences of the United States of America
Inelastic tunneling across thin amorphous films
Soviet Physics - JETP
Homeostatic plasticity in Bayesian spiking networks a Expectation Maximization with posterior constraints
Advances in Neural Information Processing Systems
Emergence of optimal decoding of population codes through STDP
Neural Computation
The organization of behavior
Inhibitory plasticity: Balance, control, and codependence
Annual Review of Neuroscience
Resistive switching: From fundamentals of nanoionic redox processes to memristive device applications
Relating STDP to BCM
Neural Computation
Nanoscale memristor device as synapse in neuromorphic systems
Nano Letters
On large-batch training for deep learning: Generalization gap and sharp minima
Cited by (86)
Astrocyte control bursting mode of spiking neuron network with memristor-implemented plasticity
2024, Chaos, Solitons and FractalsA hybrid SNN-STLSTM method for human error assessment in the high-speed railway system
2024, Advanced Engineering InformaticsMemristor-based spiking neural network with online reinforcement learning
2023, Neural Networks