Skip to main content
Registrieren
Springer Professional
SUCHE
MENÜ 1 2 3 4
nach oben

Tipp

Weitere Kapitel dieses Buchs durch Wischen aufrufen

Erschienen in:
First-Order Timed Runtime Verification Using BDDs Kapitel lesen Erstes Kapitel lesen

2020 | OriginalPaper | Buchkapitel

DeepAbstract: Neural Network Abstraction for Accelerating Verification

verfasst von : Pranav Ashok, Vahid Hashemi, Jan Křetínský, Stefanie Mohr

Erschienen in: Automated Technology for Verification and Analysis

Verlag: Springer International Publishing

Einloggen, um Zugang zu erhalten
share
TEILEN

Abstract

While abstraction is a classic tool of verification to scale it up, it is not used very often for verifying neural networks. However, it can help with the still open task of scaling existing algorithms to state-of-the-art network architectures. We introduce an abstraction framework applicable to fully-connected feed-forward neural networks based on clustering of neurons that behave similarly on some inputs. For the particular case of ReLU, we additionally provide error bounds incurred by the abstraction. We show how the abstraction reduces the size of the network, while preserving its accuracy, and how verification results on the abstract network can be transferred back to the original network.
Vorheriges Kapitel NeuralExplorer: State Space Exploration of Closed Loop Control Systems Using Neural Networks
Nächstes Kapitel Faithful and Effective Reward Schemes for Model-Free Reinforcement Learning of Omega-Regular Objectives
Fußnoten
1
Naturally, the parameter \(\alpha \) has to be less than or equal to the accuracy of \(D\).
 
Literatur
[Aba+15]
[AM18]
Akhtar, N., Mian, A.: Threat of adversarial attacks on deep learning in computer vision: a survey. In: IEEE Access, vol. 6, pp. 14410–14430 (2018)
[Ash+20]
[Bis06]
Christopher M Bishop. Pattern recognition and machine learning. springer, 2006
[CGL94]
Clarke, E.M., Grumberg, O., Long, D.E.: Model checking and abstraction. ACM Trans. Program. Lang. Syst. 16(5), 1512–1542 (1994)
[Che+17a]
Chen, X., et al.: Multi-view 3d object detection network for autonomous driving. In: CVPR (2017)
[Che+17b]
Yu, C., et al.: A Survey of Model Compression and Acceleration for Deep Neural Networks. In: CoRR abs/1710.09282 (2017)
[Cla+00]
Clarke, E.M., et al.: Counterexample-guided abstraction refinement. In: CAV (2000)
[CNR17]
Chih-Hong, C., Nührenberg, G., Ruess, H.: Maximum resilience of artificial neural networks. In: ATVA (2017)
[Den+20]
Lei, D., et al.: Model compression and hardware acceleration for neural networks: a comprehensive survey. In: Proceedings of the IEEE 108(4), 485–532 (2020)
[Don+18]
Dong, Y., et al.: Boosting adversarial attacks with momentum. In: CVPR (2018)
[Dvi+18]
Krishnamurthy, D., et al.: A dual approach to scalable verification of deep networks. In: UAI (2018)
[Ehl17]
Rüdiger, E.: Formal verification of piece-wise linear feed- forward neural networks. In: ATVA (2017)
[Geh+18]
Timon, G., et al.: Ai2: safety and robustness certification of neural networks with abstract interpretation. In: 2018 IEEE Symposium on Security and Privacy (SP) (2018)
[HMD16]
Han, S., Mao, H., Dally, W.J.: Deep compression: compressing deep neural network with pruning, trained quantization and huffman coding. In: ICLR (2016)
[HTF09]
Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The elements of statistical learning: data mining, inference, and prediction. Springer Science & Business Media, 2009
[Hua+17]
Huang, X., et al.: Safety verification of deep neural networks. In: CAV, no. 1 (2017)
[JKO18]
Julian, K.D., Kochenderfer, M.J., Owen, M.P.: Deep Neural Network Compression for Aircraft Collision Avoidance Systems. In: CoRR abs/1810.04240 (2018)
[Kat+17]
Guy, K., et al.: Reluplex: an efficient SMT solver for verifying deep neural networks. In: CAV, no. 1 (2017)
[LeC98]
[MHN13]
Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: ICML (2013)
[PA19]
Pavithra, P., Zahra, R.A.: Abstraction based output range analysis for neural networks. In: NeurIPS (2019)
[Pap+16]
Papernot, N., et al.: The limitations of deep learning in adversarial settings. In: EuroS&P. IEEE (2016)
[Ped+11]
Pedregosa, F., et al.: Scikit-learn: machine Learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
[PT10]
Luca, P., Armando, T.: An abstraction-refinement approach to verification of artificial neural networks. In: CAV (2010)
[SB15]
Suraj, S., Venkatesh Babu, R.: Data-free parameter pruning for deep neural networks. In: BMVC (2015)
[Sin+19a]
Singh, G., et al.: An abstract domain for certifying neural networks. In: Proceedings ACM Program. Lang. vol. 3.POPL, 41:1–41:30 (2019)
[Sin+19b]
Singh, G., et al.: Boosting robustness certification of neural networks. In: ICLR (Poster) (2019)
[SVS19]
Su, J., Vasconcellos Vargas, D., Sakurai, K.: One pixel attack for fooling deep neural networks. IEEE Trans. Evol. Comput. 23(5), 828–841 (2019)
[YGK19]
Elboher, Y.Y., Gottschlich, J., Katz, G.: An abstraction-based framework for neural network verification. In: arXiv e-prints, arXiv:​1910.​14574 (2019)
[ZYZ18]
Zhong, G., Yao, H., Zhou, H.: Merging neurons for structure compression of deep networks. In: ICPR (2018)
Metadaten
Titel
DeepAbstract: Neural Network Abstraction for Accelerating Verification
verfasst von
Pranav Ashok
Vahid Hashemi
Jan Křetínský
Stefanie Mohr
Copyright-Jahr
2020
Buch
Automated Technology for Verification and Analysis

Print ISBN: 978-3-030-59151-9

Electronic ISBN: 978-3-030-59152-6

Copyright-Jahr: 2020

DOI
https://doi.org/10.1007/978-3-030-59152-6_5

Premium Partner