Introduction
-
theoretically seamless applications of temporal backprop to SNNs,
-
workaround for the dead neuron issue,
-
fewer spikes for fast inference,
-
The use of NOSOs for both learning and inference minimizes the workload on the event-routing circuits in neuromorphic hardware.
-
To support the latency code, NOSONet includes minimum-latency pooling (MinPool) layers (instead of MaxPool or AvgPool) that pass the event of the minimum latency only for a given patch.
-
Each NOSO is given two symmetric thresholds (\(-\vartheta \) and \(\vartheta \)) for spiking to confine the potential distribution to the range between the symmetric thresholds.
-
BPLC + NOSO fully supports both folded and unfolded NOSONets, allowing us to use the automatic differentiation framework [31].
-
We introduce a novel learning algorithm based on the spiking latency code (BPLC + NOSO) with full derivations of the primary gradients without approximations.
-
We provide novel and essential methods for BPLC + NOSO support, such as MinPool layers and symmetric dual threshold for spiking, which greatly improve accuracy and inference efficiency.
-
We introduce a method to quickly calculate wallclock time for inference on general digital neuromorphic hardware, which allows a quick estimation of the inference delay for a given fully trained SNN.
Acronyms or Symbols | Description |
---|---|
IF | Integrate-and-fire |
LIF | Leaky integrate-and-fire |
SRM | Spike response model |
BPLC | Error-backpropagation algorithm based on latency code |
NOSO | Neurons that only spike once at most |
MinPool | Minimum-latency pooling |
SynOPS | Synaptic operations per second |
TTFS | Time to the first spike |
\(\varvec{T}_\text {lat}^{(L)}\) | Spiking latency of output neurons in the output layer L |
\(\hat{t}^{(l)}_{i}\) | Spike timing of the ith neuron in the lth layer |
\(t^{(l)}_{\text {in}, i}\) | First input spike timing for the ith neuron in the lth layer |
\(w_{ij}^{(l)}\) | Synaptic weight from the jth neuron in the (l-1)th layer |
\(u_{i}^{(l)}\) | Membrane potential of the ith neuron in the lth layer |
\(v_{j}^{(l)}\) | Membrane potential before weight multiplication of jth neuron in the lth layer |
\(N_\text {n}\) | Number of neurons in a network |
\(N_\text {c}\) | Number of cores of neuromorphic processor |
\(T_{\text {up}}\) | Time for process of multiplying the current potential by decay factor |
\(T_{\text {sop}}\) | Time for synaptic operations at each timestep |
\(T_{\text {inf}}\) | Inference delay |
Related work
Preliminaries
Latency code
Minimum-latency pooling
NOSO with dual threshold for spiking
BPLC with spike response model
Spike response model mapped onto computational graphs
Backward pass and gradients
Method | Network | Coding | Best accuracy | Average Accuracy | \(\#\)spikes \(N_{\text {sp}}\) |
---|---|---|---|---|---|
Fashion-MNIST | |||||
Ikegawa et al. [16] | 16C3-\(\{\)32C3\(\}\)*6-\(\{\)64C3\(\}\)*5 | Rate | 89.10 | – | 7156K |
Zhang et al. [46] | 16C5-P2-32C5-P2-800-128 | Temporal | 90.10 | – | – |
Zhang et al. [47] | 400-R400 | Rate | 90.13 | 90.00±0.14 | – |
Sun et al. [36] | 32C3-P2-32C3-P2-128 | Rate | 91.56 | – | 12K (only Conv) |
Cheng et al. [6] | 32C3-P2-32C3-P2-128 | Rate | 92.07 | – | – |
Mirsadeghi et al. [25] | 20C5-P2-40C5-P2-1000 | Temporal | 92.80 | – | – |
Zhang and Li. [48] | 32C5-P2-64C5-P2-1024 | Temporal | 92.83 | 92.69±0.09 | – |
Zhao et al. [49] | 32C5-P2-64C5-P2-1024 | Rate | 93.45 | 93.04±0.31 | – |
BPLC + NOSO | 32C5-P2-64C5-P2-600 | Latency | 92.47 | 92.44±0.02 | 14K±0.26K |
CIFAR-10 | |||||
Wu et al. [39] | CNN1\(^{*}\) | Rate | 85.24 | – | – |
Wu et al. [39] | CNN2\(^{**}\) | Rate | 90.53 | – | – |
Wu et al. [39] | CNN2-half-ch | Rate | 87.80 | – | 1298K |
Zhang and Li. [48] | CNN1 | Temporal | 89.22 | – | – |
Zhang and Li. [48] | CNN2 | Temporal | 91.41 | – | 308K |
Tan et al. [37] | CNN1 | Modified rate | 89.57 | – | 412K |
Tan et al. [37] | CNN2 | Modified rate | 90.13 | – | 342K |
Zhao et al. [49] | CNN3\(^{***}\) | Rate | 90.93 | – | – |
Lee et al. [23] | ResNet11 | Rate | 90.95 | – | 1530K |
BPLC + NOSO | CNN4\(^{****}\) | Latency | 89.77 | 89.37±0.25 | 142K±1.86K |
Experiments
Classification accuracy and the number of spikes for inference
Minimum-latency pooling versus MaxPool
Effect of symmetric dual threshold on potential distribution
Discussion
Conclusion and outlook
-
Scalability confirmation: Although the viability of BPLC + NOSO was identified, its applicability to deeper SNNs on more complex datasets should be confirmed. Such datasets include not only static image datasets like ImageNet [33] but also event datasets like CIFAR10-DVS [24] and DVS128 Gesture [1]. Given that the number of spikes is severely capped, BPLC + NOSO on event datasets in particular might be challenging.
-
Hyperparameter fine-tuning: To further increase the classification accuracy, the hyperparameters should be fine-tuned using optimization techniques.
-
Weight quantization: BPLC + NOSO is based on full-precision (32b FP) weights. However, the viability of BPLC + NOSO with reduced precision weights should be confirmed to improve the efficiency in memory use. This may need an additional weight-quantization algorithm in conjunction with BPLC + NOSO like CBP [18].