Supervised learning in multilayer spiking neural networks with inner products of spike trains

Abstract

Recent advances in neurosciences have revealed that neural information in the brain is encoded through precisely timed spike trains, not only through the neural firing rate. This paper presents a new supervised, multi-spike learning algorithm for multilayer spiking neural networks, which can implement the complex spatio-temporal pattern learning of spike trains. The proposed algorithm firstly defines inner product operators to mathematically describe and manipulate spike trains, and then solves the problems of error function construction and backpropagation among multiple output spikes during learning. The algorithm is successfully applied to different temporal tasks, such as learning sequences of spikes and nonlinear pattern classification problems. The experimental results show that the proposed algorithm has higher learning accuracy and efficiency than the Multi-ReSuMe learning algorithm. It is effective for solving complex spatio-temporal pattern learning problems.

Introduction

Traditional artificial neural networks (ANNs) encode information by the firing rate of the biological neurons, and the outputs of neurons are generally expressed as analog variables in the given interval. Their learning algorithms are to minimize a selected cost or error function (a measure of the difference between the network outputs and the desired outputs) by adjusting the measures of synaptic strength. They mainly depend on the real values of neuronal outputs [1], such as the widely-used backpropagation (BP) training algorithm [2], [3]. However, experimental evidence from the field of neuroscience suggests that neural systems encode information through the precise timing of spikes, not only through the neural firing rate [4], [5]. Using a biologically plausible spiking neuron model [6], [7] as the basic unit for constructing spiking neural networks (SNNs), they encode and process neural information through the precisely timed spike trains. SNNs are often referred to as the new generation of neural networks [8], [9]. They have more powerful computing capacity to simulate a variety of neuronal signals and approximate any continuous function [10], [11], and have been shown to be suitable tools for the processing of spatio-temporal information.

Supervised learning in ANNs involves a mechanism of providing the desired outputs with the corresponding inputs [12]. The network then processes the inputs and compares its resulting outputs against the desired outputs. Errors are calculated to control the synaptic weight adjustment. This process occurs over and over until the synaptic weights converge to certain values. The set of data that enables the training is called the training set. When the sample conditions changed, synaptic weights can be modified through supervised learning to adapt to the new environment. Experimental studies have shown that supervised learning exists in the biological nervous system, especially in the sensorimotor networks and sensory system [13], [14], [15], but there is no clear conclusion to explain how biological neurons realize this process. The purpose of supervised learning with temporal encoding for spiking neurons is to make the neurons emit arbitrary spike trains in response to given synaptic inputs. At present, researchers have conducted many studies on the supervised learning in SNNs [16], [17], and achieved some results, but many problems remain unsolved. The supervised learning algorithms for SNNs proposed in recent years can be roughly divided into three categories: (1) supervised learning algorithms based on gradient descent, (2) supervised learning algorithms based on a synaptic plasticity mechanism, and (3) supervised learning algorithms based on the convolution of spike trains.

Supervised learning algorithms based on gradient descent use gradient computation and error backpropagation for adjusting the synaptic weights, and ultimately minimize the error function that indicates the deviation between the actual and the desired output spikes. Bohte et al. [18] first proposed a backpropagation training algorithm for feedforward SNNs, called SpikeProp, similar in concept to the BP algorithm developed for traditional ANNs [2]. The spike response model (SRM) [19] is used in this algorithm. In SRM, the neuronal potential is represented by analytical expression. To overcome the discontinuity of the internal state variable caused by spike firing, all neurons in the network can fire only one single spike. Xin and Embrechts [20] presented a method with simple momentum that accelerates the convergence speed of the SpikeProp algorithm. In addition, McKennoch et al. proposed RProp and QuickProp algorithms [21] with faster convergence, and further extended SpikeProp to a class of nonlinear neuron models and constructed a BP algorithm in Theta neuron networks [22]. Fang et al. [23] proposed a learning rate adaptive method in which the learning rate dynamically changes in the learning process. However, all of the above algorithms encode information with a single spike, a limitation that means they cannot be effective for solving complex problems. A more important extension of SpikeProp was presented by Booij and Nguyen [24]. Their algorithm allows the neurons in the input and hidden layers to fire multiple spikes, but only the first spike in the output layer is considered; this minimizes the time difference between the actual output spike and the target spike. Similarly, Ghosh-Dastidar and Adeli [25] put forward a BP learning algorithm named Multi-SpikeProp, with derivations of the learning rule based on the chain rule for a multi-spiking network model. Multi-SpikeProp was applied to the standard XOR problem and the Fisher Iris and EEG classification problems, and experimental results show that the algorithm has higher classification accuracy than the SpikeProp algorithm. Recently, Xu et al. [26] have extended the Multi-SpikeProp algorithm to allow neurons to fire multiple spikes in all layers. That is, the algorithm can implement the complex spatio-temporal pattern learning of spike trains. The experimental results show that this algorithm has higher learning accuracy for a large number of output spikes. Florian [27] introduced two supervised learning rules for spiking neurons with temporal coding of information (Chronotrons), and an E-learning rule based on gradient descent provides high memory capacity. But the E-learning rule is only suitable for a single neuron or single layer network. Supervised learning algorithms based on gradient descent have been developed mostly for simple neuron models that require the analytical expression of state variables. These algorithms cannot be applied to various neuron models.

Supervised learning algorithms based on a synaptic plasticity mechanism, in contrast, aim at modeling the learning rule from the correlation of spike firing times of the presynaptic neuron and postsynaptic neuron, which are more biologically plausible learning algorithms. In fact, the spike train can not only cause persistent changes of synaptic strength, but it also satisfies the spike-timing-dependent plasticity (STDP) mechanism [28], [29]. Based on the STDP learning rule, many researchers have proposed various supervised learning algorithms suitable for SNNs. Legenstein et al. [30] presented a supervised Hebbian learning algorithm for spiking neurons, based on injecting external input current to make the learning neurons fire in a specific target spike train. Combining the Bienenstock-Cooper-Munro (BCM) learning rule with the STDP mechanism, a synaptic weight association training (SWAT) algorithm for SNNs is proposed [31], which yields a unimodal synaptic weight distribution where weight stabilization is achieved using the sliding threshold associated with the BCM model after a period of training. The chronotron I-Learning rule [27] changes the synaptic weights depending on the synaptic currents at the timings of actual and target output spikes. The algorithm has a high biological plausibility, but it can only be applied to single layer networks. Ponulak et al. [32], [33] proposed the ReSuMe algorithm, which adjusts the synaptic weights according to STDP and anti-STDP processes and is suitable for various types of spiking neuron models. However, the algorithm can only be applied to single layer networks or train readouts for reservoir networks. Recently, Sporea and Grüning [34] extended the ReSuMe algorithm to multilayer feedforward SNNs using backpropagation of the network error. The weights are updated according to STDP and anti-STDP rules, and the neurons in every layer can fire multiple spikes. This algorithm is named Multi-ReSuMe. Simulation experiments show that the algorithm can be successfully applied to various complex classification problems and permits precise spike train encoding.

The main idea of the last class of supervised learning algorithms is the definition of convolution of the spike trains to design the proper learning rule. A spike train contains an abstraction of neurophysiological recordings [35], which is a simply sequence of ordered spike times. For convenience of analysis and calculation, we can select a specific convolution kernel to transform spike trains into continuous functions so that common mathematical operators can be performed on them. Through the convolution calculation of a spike train based on kernel function, the spike train can be interpreted as a specific neural physiological signal, such as neuronal postsynaptic potential or spike firing intensity function [36]. Evaluating the relationship between spike trains by the definition of convolution kernel, a supervised learning algorithm for SNNs can be constructed based on the difference of the kernelized spike trains. Carnell and Richardson [37] expanded the set of spike trains into a vector space, then applied linear algebra methods to implement the spatio-temporal pattern learning of spike trains. Mohemmed et al. [38], [39] proposed a SPAN algorithm based on a Hebbian interpretation of the Widrow-Hoff rule and kernel function convolution. Inspired by the SPAN algorithm, Yu et al. [40], [41] proposed a PSD supervised learning rule that can be used to train neurons to associate an input spatio-temporal spike pattern with a desired spike train. Unlike the SPAN method that requires spike convolution on all the spike trains of the input, the desired output and the actual output, the PSD learning rule only convolves the input spike trains. However, none of these three algorithms implements the error backpropagation mechanism, and all can be applied only to a single neuron or single layer SNNs.

For SNNs, input and output information is encoded through precisely timed spike trains, not only through the neural firing rate. In addition, the internal state variables of spiking neurons and error function do not satisfy the continuous differentiability. So, traditional learning algorithms of ANNs, especially the BP algorithm, cannot be used directly, and the formulation of efficient supervised learning algorithms for SNNs is a very challenging problem. In this paper, we present a new supervised learning algorithm for feedforward SNNs with multiple layers. We refer to this algorithm as Multi-STIP for multilayer SNNs learning with spike train inner products. The Multi-STIP learning algorithm combines the mechanism of error backpropagation, constructing a novel error function of spike trains and spanning to multiple layers, with a synaptic weight learning rule based on the difference of inner products of spike trains, which can be applied to neurons firing multiple spikes in all layers.

The rest of this paper is organized as follows. In Section 2 we analyze and define the inner products of spike trains. In Section 3 we construct the error function and derive the learning rule based on the inner products of spike trains for multilayer feedforward SNNs. In Section 4 the flexibility and power of feedforward SNNs trained with our algorithm are showcased by a spike sequence learning problem and a nonlinear pattern classification task. The discussion and conclusion are presented in Section 5.