Elsevier

Neural Networks

Volume 134, February 2021, Pages 64-75
Neural Networks

Necessary conditions for STDP-based pattern recognition learning in a memristive spiking neural network

https://doi.org/10.1016/j.neunet.2020.11.005Get rights and content

Highlights

  • Supporting correlations in activities of neurons is a near-optimal learning policy.

  • Binary clusterization can be a benchmark for tuning parameters of a rate-coding SNN.

  • Shaping memristive STDP window for binary clusterization helps in more complex tasks.

  • Nanocomposite LiNbO3-based memristors are suitable for always-on learning SNNs.

Abstract

This work is aimed to study experimental and theoretical approaches for searching effective local training rules for unsupervised pattern recognition by high-performance memristor-based Spiking Neural Networks (SNNs). First, the possibility of weight change using Spike-Timing-Dependent Plasticity (STDP) is demonstrated with a pair of hardware analog neurons connected through a (CoFeB)x(LiNbO3)1x nanocomposite memristor. Next, the learning convergence to a solution of binary clusterization task is analyzed in a wide range of memristive STDP parameters for a single-layer fully connected feedforward SNN. The memristive STDP behavior supplying convergence in this simple task is shown also to provide it in the handwritten digit recognition domain by the more complex SNN architecture with a Winner-Take-All competition between neurons. To investigate basic conditions necessary for training convergence, an original probabilistic generative model of a rate-based single-layer network with independent or competing neurons is built and thoroughly analyzed. The main result is a statement of “correlation growth-anticorrelation decay” principle which prompts near-optimal policy to configure model parameters. This principle is in line with requiring the binary clusterization convergence which can be defined as the necessary condition for optimal learning and used as the simple benchmark for tuning parameters of various neural network realizations with population-rate information coding. At last, a heuristic algorithm is described to experimentally find out the convergence conditions in a memristive SNN, including robustness to a device variability. Due to the generality of the proposed approach, it can be applied to a wide range of memristors and neurons of software- or hardware-based rate-coding single-layer SNNs when searching for local rules that ensure their unsupervised learning convergence in a pattern recognition task domain.

Introduction

The development of spiking neural networks (SNNs) has been mostly driven by the hope of replication the extraordinary low energy consumption and high computing efficiency of biological systems in a solution of so-called anthropogenic tasks (text and speech processing, pattern recognition, decision making, prediction, etc.) (Gerstner & Kistler, 2002). SNNs comprise layers (typically input, hidden and output) of artificial neurons (that collect, integrate and fire spikes) and connecting them synapses that represent strength (or weight) of the connection. Synaptic weights are tunable during training. Circuits based on complementary metal oxide semiconductor (CMOS) elements have been successfully used to implement SNNs, e.g. TrueNorth (Merolla et al., 2014), SpiNNaker (Furber, Galluppi, Temple, & Plana, 2014), Loihi (Davies et al., 2018), and others (Chen, Kumar, Sumbul, Knag, & Krishnamurthy, 2019). However, as the CMOS digital devices were not created or optimized for the purposes of neuromorphic (brain-inspired) computing, they do not faithfully emulate synapses and hence lack the intrinsic hardware training capability. In this respect memristors (“resistors with memory”) can emulate synapses (Strukov, Snider, Stewart, & Williams, 2008) and partially neurons (Pickett, Medeiros-Ribeiro, & Williams,2013) more faithfully because they share fundamentally similar operation mechanisms with their bio-counterparts: both mechanisms are closely associated with ion drift or diffusion and analog character of signal processing (Xia & Yang, 2019). Moreover, memristor-based neuromorphic networks with organization of the weights in a crossbar architecture have an ability to perform massively parallel and highly energy efficient vector-by-matrix multiplication directly at the site where data is stored, using the Kirchhoff electric current summation rules (Ielmini & Waser, 2016).

There are two most common types of memristor-based neural networks, namely deep neural networks (DNNs) and SNNs. Promising DNN schemes, such as multilayer perceptrons (Choi et al., 2017, Emelyanov et al., 2016, Li et al., 2018, Merrikh-Bayat et al., 2018, Mikhaylov et al., 2018, Silva et al., 2020), sparse coding networks (Cai et al., 2019), long short-term memory (Li et al., 2019), networks with reinforcement learning (Wang et al., 2019) and others (Akhmetov and James, 2019, Dowling et al., 2020, Sun et al., 2019) have been presented with the use of analog memristive weights between artificial software- or hardware-based neurons. However, training in these networks typically relies on a various kinds of error back-propagation algorithm, the efficient implementation of which is challenging due to the necessity of a global (operating at the level of a whole network) method for calculation of weight updates and the requirement for memristors with high cycle-to-cycle and device-to-device reproducibility (Del Valle, Ramírez, Rozenberg, & Schuller, 2018).

SNNs, in contrast to DNNs, encode information in rates and timing of spikes that gives more subtle dynamic responses and richer representations of an object features applied to the network input (Lobo, Del Ser, Bifet, & Kasabov, 2020). This is especially evident in the processing of temporal sequences such as speech and video, in the generation of control signals for mobile and robotic devices (Xia & Yang, 2019). Additionally, SNN training is based on local rules of synaptic weight update, requiring only information from pre- and post-synaptic neurons. This could be a significant advantage for compact and low power implementations of a real-time training, since the weight update does not require calculating each time the error gradients for neurons in the deeper layers of a network, and scaling to more complex networks (Merolla et al., 2014). Local rules potentially could provide correct modification of recurrent (backward and intra-layer) connections, including interneuron inhibition, or competition between neurons, which can significantly reduce requirements to the size of labeled training datasets (Demin & Nekhaev, 2018). However, despite their great potential, the computational abilities of SNNs have not been demonstrated as much as those of DNNs, mainly because of great but insufficient study of effective operating algorithms and learning rules, especially in hardware terms.

One of the most perspective local SNN learning rules is a Spike-Timing Dependent Plasticity (STDP), which is a type of Hebbian learning (Caporale & Dan, 2008) and can be successfully realized with different memristors (Ielmini and Waser, 2016, Jo et al., 2010, Kim et al., 2015, Kim et al., 2017, Minnekhanov et al., 2019, Prezioso et al., 2016). According to the basic pairwise model of STDP, a synaptic weight increases if the pre-synaptic neuron generates a spike just before the post-synaptic one (indicating a causal relationship), and a synaptic weight decreases if the post- synaptic neuron spikes just before the pre- synaptic one (reflecting a non-causal relationship) (Bi & Poo, 1998). Memristor implementation of STDP could also be modulated via heterosynaptic plasticity or dopamine-like assistance mechanisms which allows the realization of bio-inspired reinforcement learning algorithms (Maier et al., 2016, Nikiruy, Emelyanov et al., 2019). This makes the STDP mechanism the foundation both for supervised and unsupervised learning of memristor-based SNNs (Covi et al., 2016).

Although quite a lot number of works implementing STDP learning in memristor-based SNNs (Covi et al., 2018, Emelyanov et al., 2020, Nikiruy, Emelyanov, Rylkov, Sitnikov and Demin, 2019, Prezioso et al., 2018, Serb et al., 2016, Wang et al., 2018) and their simulation models (Ambrogio et al., 2016, Bill and Legenstein, 2014, Boyn et al., 2017, Lobov et al., 2020, Moraitis et al., 2017, Qu et al., 2020, Querlioz et al., 2013) have been demonstrated, it is not totally clear how STDP experimentally realized on memristors deals with training convergence issues. What parameters of a so-called “STDP window” (weight update dependence on inter-spike time intervals) and memristor characteristics are appropriate for STDP-based training of memristive networks to converge to an expected solution? One of the early thorough research in Querlioz et al. (2013) showed in a simulation that learning of a single-layer SNN with a quite simple exponential model of memristor weight update is largely robust with respect to the high enough variations of the minimum and maximum values of memristor conductivities, increments in the weight update, the limited resolution of the memristor conductance and the read disturb effect, the stimulus coding schemes, and the presence of some noise spikes at the inputs. At the same time, it is not yet clear how these remarkable results map to more complex forms of STDP and corresponding STDP window shapes observed in experiments with real memristors and hardware neurons, how the variations of experimentally observed STDP window parameters influence the direction of weight change under specific stimuli, and, at last, how these issues correlate with an accuracy of SNN solving some pattern recognition task. The present work addresses these issues from experimental, simulation and theoretical points of view that could be of high importance in practice when searching for memristor devices and pulse shapes appropriate for SNN learning convergence.

All physical experiments were done with (CoFeB)x(LiNbO3)1x nanocomposite based memristors that demonstrate very appealing characteristics (Nikiruy, Emelyanov et al., 2019, Rylkov et al., 2018).

Section snippets

Analog neuron

To generate bi-triangular voltage pulses (spikes) with adjustable amplitude and duration parameters, a special leakyintegrate-and-fire (LIF) neuron scheme was developed. The main feature of this circuit differing it from many other hardware realizations of neurons (Acciarito et al., 2017, Ambrogio et al., 2016, Mahalanabis et al., 2016, Serrano-Gotarredona et al., 2013, Wu et al., 2015) is the generation of a bi-triangular spike and its transmission in both output and input directions of the

Memristive STDP

The NC memristors demonstrate resistive switching with the resistance ratio ROFFRON more than 103 and retention time more than 104s (ROFF, RON are the memristor resistances in OFF and ON resistive states, respectively). Typical I-V curves, endurance and resistive state time dependencies are presented in Supplementary Figs. S7, S8. The switching mechanism is based on the conductive filaments formation and destruction due to electromigration of oxygen vacancies controlled by the percolation

Spike-rate dependent plasticity

The simulation results show that the convergence of thememristor-based single-layer SNN training can be achieved in a wide, but not in the entire range of parameters of the STDP window model. It can be seen that the ratios of the amplitudes (A+,A), widths (μ+,μ) and window branch relaxation times (τ+,τ) are much more important than their absolute values. An influence of these ratios can most clearly be understood in a ‘post-pre-post’ spike triplet pairing scheme, when the two weight updates

Conclusions

In conclusion, we have experimentally found and theoretically proved the necessary conditions for guaranteeing the near-optimal clusterization learning in population-rate coding neural networks with either spike- or rate-based neurons. These conditions rely on the formulated ‘correlation growth-anticorrelation decay’ principle which provides the potentiation of weights of co-active input–output neuron pairs and weight decay between pre- and postsynaptic neurons that are in antiphase of their

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This work was supported by the Russian Foundation for Basic Research (Grant No. 18-29-23041) in part of analog neuron realization, SNN simulation, and building the theoretical model, and Russian Science Foundation (Grant No. 16-19-10233) in part of STDP implementation using (CoFeB)x(LiNbO3)1x nanocomposite memristor. Measurements were carried out on the equipment of the Resource centers under support of the NRC “Kurchatov Institute” (#1055).

Authors are thankful to Prof. A.V. Sitnikov

References (73)

  • CaiF. et al.

    A fully integrated reprogrammable memristor–CMOS system for efficient multiply–accumulate operations

    Nature Electronics

    (2019)
  • CaporaleN. et al.

    Spike timing–dependent Plasticity: A Hebbian learning rule

    Annual Review of Neuroscience

    (2008)
  • ChenG.K. et al.

    A 4096-Neuron 1M-Synapse 3.8-pJ/SOP spiking neural network with On-Chip STDP Learning and Sparse Weights in 10-nm FinFET CMOS

    IEEE Journal of Solid-State Circuits

    (2019)
  • ChiccaE. et al.

    A recipe for creating ideal hybrid memristive-CMOS neuromorphic processing systems

    Applied Physics Letters

    (2020)
  • ChoiS. et al.

    Experimental demonstration of feature extraction and dimensionality reduction using memristor networks

    Nano Letters

    (2017)
  • CoviE. et al.

    Analog memristive synapse in spiking networks implementing unsupervised learning

    Frontiers in Neuroscience

    (2016)
  • CoviE. et al.

    Spike-driven threshold-based learning with memristive synapses and neuromorphic silicon neurons

    Journal of Physics D (Applied Physics)

    (2018)
  • DaviesM. et al.

    Loihi: A neuromorphic Manycore processor with On-Chip Learning

    IEEE Micro

    (2018)
  • Del ValleJ. et al.

    Challenges in materials and devices for resistive-switching-based neuromorphic computing

    Journal of Applied Physics

    (2018)
  • DeminV.A. et al.

    Recurrent spiking neural network learning based on a competitive maximization of neuronal activity

    Frontiers in Neuroinformatics

    (2018)
  • DempsterA.P. et al.

    Maximum likelihood from incomplete data via the EM algorithm

    The Journal of the Royal Statistical Society, Series B

    (1977)
  • DowlingV.J. et al.

    Probabilistic Memristive networks: Application of a Master Equation to Networks of binary ReRAM cells

    (2020)
  • EmelyanovA.V. et al.

    First steps towards the realization of a double layer perceptron based on organic memristive devices

    Advances

    (2016)
  • EmelyanovA.V. et al.

    Self-adaptive STDP-based learning of a spiking neuron with nanocomposite memristive weights

    Nanotechnology

    (2020)
  • FurberS.B. et al.

    The SpiNNaker project

    Proceedings of the IEEE

    (2014)
  • GerstnerW. et al.

    Spiking neuron models

    (2002)
  • GjorgjievaJ. et al.

    A triplet spike-timing-dependent plasticity model generalizes the Bienenstock-Cooper-Munro rule to higher-order spatiotemporal correlations

    Proceedings of the National Academy of Sciences of the United States of America

    (2011)
  • GlazmanL.I. et al.

    Inelastic tunneling across thin amorphous films

    Soviet Physics - JETP

    (1988)
  • HabenschussS. et al.

    Homeostatic plasticity in Bayesian spiking networks a Expectation Maximization with posterior constraints

    Advances in Neural Information Processing Systems

    (2012)
  • HabenschussS. et al.

    Emergence of optimal decoding of population codes through STDP

    Neural Computation

    (2013)
  • HebbD.O.

    The organization of behavior

    (1949)
  • HennequinG. et al.

    Inhibitory plasticity: Balance, control, and codependence

    Annual Review of Neuroscience

    (2017)
  • IelminiD. et al.

    Resistive switching: From fundamentals of nanoionic redox processes to memristive device applications

    (2016)
  • IzhikevichE.M. et al.

    Relating STDP to BCM

    Neural Computation

    (2003)
  • JoS.H. et al.

    Nanoscale memristor device as synapse in neuromorphic systems

    Nano Letters

    (2010)
  • KeskarN.S. et al.

    On large-batch training for deep learning: Generalization gap and sharp minima

    (2016)
  • Cited by (86)

    View all citing articles on Scopus
    View full text