Skip to main content

2020 | OriginalPaper | Buchkapitel

Reliability Enhancement of Neural Networks via Neuron-Level Vulnerability Quantization

verfasst von : Keyao Li, Jing Wang, Xin Fu, Xiufeng Sui, Weigong Zhang

Erschienen in: Algorithms and Architectures for Parallel Processing

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Neural networks are increasingly used in recognition, mining and autonomous driving. However, for safety-critical applications, such as autonomous driving, the reliability of NN is an important area that remains largely unexplored. Fortunately, NN itself has fault-tolerance capability, especially, different neurons have different fault-tolerance capability. Thus applying uniform error protection mechanism while ignore this important feature will lead to unnecessary energy and performance overheads. In this paper, we propose a neuron vulnerability factor (NVF) quantifying the neural network vulnerability to soft error, which could provide a good guidance for error-tolerant techniques in NN. Based on NVF, we propose a computation scheduling scheme to reduce the lifetime of neurons with high NVF. The experiment results show that our proposed scheme can improve the accuracy of the neural network by 12% on average, and greatly reduce the fault-tolerant overhead.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Sangchoolie, B., Pattabiraman, K., Karlsson, J.: One bit is (not) enough: an empirical study of the impact of single and multiple bit-flip errors. In: International Conference on Dependable Systems and Networks (2017) Sangchoolie, B., Pattabiraman, K., Karlsson, J.: One bit is (not) enough: an empirical study of the impact of single and multiple bit-flip errors. In: International Conference on Dependable Systems and Networks (2017)
2.
Zurück zum Zitat Azizimazreah, A., et al.: Tolerating soft errors in deep learning accelerators with reliable on-chip memory designs. In: IEEE International Conference on Networking, Architecture and Storage (NAS) (2018) Azizimazreah, A., et al.: Tolerating soft errors in deep learning accelerators with reliable on-chip memory designs. In: IEEE International Conference on Networking, Architecture and Storage (NAS) (2018)
3.
Zurück zum Zitat Karlsson, J., Liden, P., Dahlgren, P., Johansson, R., Gunneflo, U.: Using heavy-ion radiation to validate fault-handling mechanisms. In: MICRO (1994) Karlsson, J., Liden, P., Dahlgren, P., Johansson, R., Gunneflo, U.: Using heavy-ion radiation to validate fault-handling mechanisms. In: MICRO (1994)
4.
Zurück zum Zitat Zhang, Q., Xu, Q.: Approxit: a quality management framework of approximate computing for iterative methods. IEEE Trans. Comput.-Aided Des. Integr. Circ. Syst. (2017) Zhang, Q., Xu, Q.: Approxit: a quality management framework of approximate computing for iterative methods. IEEE Trans. Comput.-Aided Des. Integr. Circ. Syst. (2017)
5.
Zurück zum Zitat Chen, X., Chen, D.Z., Hu, X.S.: moDNN: memory optimal DNN training on GPUs. In: DATE (2018) Chen, X., Chen, D.Z., Hu, X.S.: moDNN: memory optimal DNN training on GPUs. In: DATE (2018)
6.
Zurück zum Zitat Ma, Y., et al.: Optimizing loop operation and dataflow in FPGA acceleration of deep convolutional neural networks. In: FPGA (2017) Ma, Y., et al.: Optimizing loop operation and dataflow in FPGA acceleration of deep convolutional neural networks. In: FPGA (2017)
7.
Zurück zum Zitat Savino, A., Vallero, A., Carlo, S.D.: ReDO: cross-layer multi-objective design-exploration framework for efficient soft error resilient systems. IEEE Trans. Comput. 67, 1462–1477 (2018)MathSciNetCrossRef Savino, A., Vallero, A., Carlo, S.D.: ReDO: cross-layer multi-objective design-exploration framework for efficient soft error resilient systems. IEEE Trans. Comput. 67, 1462–1477 (2018)MathSciNetCrossRef
8.
Zurück zum Zitat Reagen, B., et al.: Minerva: enabling low-power, highly-accurate deep neural network accelerators. In: ISCA (2016) Reagen, B., et al.: Minerva: enabling low-power, highly-accurate deep neural network accelerators. In: ISCA (2016)
9.
Zurück zum Zitat Venkataramani, S., Ranjan, A., Roy, K., Raghunathan, A.: AxNN: energy-efficient neuromorphic systems using approximate computing. In: ISLPED (2014) Venkataramani, S., Ranjan, A., Roy, K., Raghunathan, A.: AxNN: energy-efficient neuromorphic systems using approximate computing. In: ISLPED (2014)
10.
Zurück zum Zitat Zhang, Q., Wang, T., Tian, Y., Yuan, F., Xu, Q.: ApproxANN: an approximate computing framework for artificial neural network. In: DATE (2015) Zhang, Q., Wang, T., Tian, Y., Yuan, F., Xu, Q.: ApproxANN: an approximate computing framework for artificial neural network. In: DATE (2015)
11.
Zurück zum Zitat Mukherjee, S.S., Weaver, C., Emer, J., Reinhardt, S.K.: A systematic methodology to compute the are ehitectural vulnerability factors for a hish performance microprocessor. In: MICRO (2003) Mukherjee, S.S., Weaver, C., Emer, J., Reinhardt, S.K.: A systematic methodology to compute the are ehitectural vulnerability factors for a hish performance microprocessor. In: MICRO (2003)
Metadaten
Titel
Reliability Enhancement of Neural Networks via Neuron-Level Vulnerability Quantization
verfasst von
Keyao Li
Jing Wang
Xin Fu
Xiufeng Sui
Weigong Zhang
Copyright-Jahr
2020
DOI
https://doi.org/10.1007/978-3-030-38961-1_24