2016 | OriginalPaper | Buchkapitel
Tipp
Weitere Kapitel dieses Buchs durch Wischen aufrufen
Erschienen in:
Neural Information Processing
The Spiking Neural Network (SNN) has not achieved the recognition/classification performance of its non-spiking competitor, the Artificial Neural Network(ANN), particularly when used in deep neural networks. The mapping of a well-trained ANN to an SNN is a hot topic in this field, especially using spiking neurons with biological characteristics. This paper proposes a new biologically-inspired activation function, Noisy Softplus, which is well-matched to the response function of LIF (Leaky Integrate-and-Fire) neurons. A convolutional network (ConvNet) was trained on the MNIST database with Noisy Softplus units and converted to an SNN while maintaining a close classification accuracy. This result demonstrates the equivalent recognition capability of the more biologically-realistic SNNs and bring biological features to the activation units in ANNs.
Bitte loggen Sie sich ein, um Zugang zu diesem Inhalt zu erhalten
Sie möchten Zugang zu diesem Inhalt erhalten? Dann informieren Sie sich jetzt über unsere Produkte:
Anzeige
1.
Buesing, L., Bill, J., Nessler, B., Maass, W.: Neural dynamics as sampling: a model for stochastic computation in recurrent networks of spiking neurons. PLoS Comput. Biol.
7(11), e1002211 (2011)
MathSciNetCrossRef
2.
Cao, Y., Chen, Y., Khosla, D.: Spiking deep convolutional neural networks for energy-efficient object recognition. Int. J. Comput. Vis.
113(1), 54–66 (2015)
MathSciNetCrossRef
3.
Davison, A.P., Brüderle, D., Eppler, J., Kremkow, J., Muller, E., Pecevski, D., Perrinet, L., Yger, P.: PyNN: a common interface for neuronal network simulators. Front. Neuroinform.
2, 1–11 (2008)
CrossRef
4.
Diehl, P.U., Neil, D., Binas, J., Cook, M., Liu, S.C., Pfeiffer, M.: Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In: International Joint Conference on Neural Networks (IJCNN). IEEE (2015)
5.
Diehl, P.U., Zarrella, G., Cassidy, A., Pedroni, B.U., Neftci, E.: Conversion of artificial recurrent neural networks to spiking neural networks for low-power neuromorphic hardware. arXiv preprint (2016)
6.
Furber, S.B., Galluppi, F., Temple, S., Plana, L., et al.: The SpiNNaker project. Proc. IEEE
102(5), 652–665 (2014)
CrossRef
7.
Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: International Conference on Artificial Intelligence and Statistics, pp. 315–323 (2011)
8.
He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1026–1034 (2015)
9.
Hunsberger, E., Eliasmith, C.: Spiking deep networks with LIF neurons. arXiv preprint (2015)
10.
Jug, F., Lengler, J., Krautz, C., Steger, A.: Spiking networks and their rate-based equivalents: does it make sense to use Siegert neurons? Swiss Soc. Neurosci. (2012).
https://www1.ethz.ch/cadmo/as/people/members/fjug/personal_home/preprints/2012_SiegertAbstract.pdf
11.
Liu, Q., Garibaldia, P.-G., Stromatias, E., Serrano-Gotarredona, T., Furber, S.: Benchmarking spike-based visual recognition: a dataset and evaluation. Front. Neurosci. (2016, under review)
12.
Merolla, P.A., Arthur, J.V., Alvarez-Icaza, R., Cassidy, A.S., Sawada, J., Akopyan, F., Jackson, B.L., Imam, N., Guo, C., Nakamura, Y., et al.: A million spiking-neuron integrated circuit with a scalable communication network and interface. Science
345(6197), 668–673 (2014)
CrossRef
13.
Neftci, E., Das, S., Pedroni, B., Kreutz-Delgado, K., Cauwenberghs, G.: Event-driven contrastive divergence for spiking neuromorphic systems. Front. Neurosci.
7, 272 (2013)
14.
O’Connor, P., Welling, M.: Deep spiking networks. arXiv preprint (2016)
15.
Stromatias, E., Neil, D., Galluppi, F., Pfeiffer, M., Liu, S.C., Furber, S.: Scalable energy-efficient, low-latency implementations of trained spiking deep belief networks on SpiNNaker. In: International Joint Conference on Neural Networks (IJCNN). IEEE (2015)
- Titel
- Noisy Softplus: A Biology Inspired Activation Function
- DOI
- https://doi.org/10.1007/978-3-319-46681-1_49
- Autoren:
-
Qian Liu
Steve Furber
- Sequenznummer
- 49