Weitere Kapitel dieses Buchs durch Wischen aufrufen

Erschienen in:

2020 | OriginalPaper | Buchkapitel

DeepAbstract: Neural Network Abstraction for Accelerating Verification

verfasst von : Pranav Ashok, Vahid Hashemi, Jan Křetínský, Stefanie Mohr

Erschienen in:

Abstract

While abstraction is a classic tool of verification to scale it up, it is not used very often for verifying neural networks. However, it can help with the still open task of scaling existing algorithms to state-of-the-art network architectures. We introduce an abstraction framework applicable to fully-connected feed-forward neural networks based on clustering of neurons that behave similarly on some inputs. For the particular case of ReLU, we additionally provide error bounds incurred by the abstraction. We show how the abstraction reduces the size of the network, while preserving its accuracy, and how verification results on the abstract network can be transferred back to the original network.
Fußnoten
1
Naturally, the parameter $$\alpha$$ has to be less than or equal to the accuracy of $$D$$.

Literatur
[Aba+15]
Abadi, M., et al.: TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Software available from tensorflow.org. (2015). https://​www.​tensorflow.​org/​
[AM18]
Akhtar, N., Mian, A.: Threat of adversarial attacks on deep learning in computer vision: a survey. In: IEEE Access, vol. 6, pp. 14410–14430 (2018)
[Ash+20]
Ashok, P., et al.: DeepAbstract: neural network abstraction for accelerating verification. Technical report (2020). arXiv:​ 2006.​13735 [cs.LO]
[Bis06]
Christopher M Bishop. Pattern recognition and machine learning. springer, 2006
[CGL94]
Clarke, E.M., Grumberg, O., Long, D.E.: Model checking and abstraction. ACM Trans. Program. Lang. Syst. 16(5), 1512–1542 (1994)
[Che+17a]
Chen, X., et al.: Multi-view 3d object detection network for autonomous driving. In: CVPR (2017)
[Che+17b]
Yu, C., et al.: A Survey of Model Compression and Acceleration for Deep Neural Networks. In: CoRR abs/1710.09282 (2017)
[Cla+00]
Clarke, E.M., et al.: Counterexample-guided abstraction refinement. In: CAV (2000)
[CNR17]
Chih-Hong, C., Nührenberg, G., Ruess, H.: Maximum resilience of artificial neural networks. In: ATVA (2017)
[Den+20]
Lei, D., et al.: Model compression and hardware acceleration for neural networks: a comprehensive survey. In: Proceedings of the IEEE 108(4), 485–532 (2020)
[Don+18]
Dong, Y., et al.: Boosting adversarial attacks with momentum. In: CVPR (2018)
[Dvi+18]
Krishnamurthy, D., et al.: A dual approach to scalable verification of deep networks. In: UAI (2018)
[Ehl17]
Rüdiger, E.: Formal verification of piece-wise linear feed- forward neural networks. In: ATVA (2017)
[Geh+18]
Timon, G., et al.: Ai2: safety and robustness certification of neural networks with abstract interpretation. In: 2018 IEEE Symposium on Security and Privacy (SP) (2018)
[HMD16]
Han, S., Mao, H., Dally, W.J.: Deep compression: compressing deep neural network with pruning, trained quantization and huffman coding. In: ICLR (2016)
[HTF09]
Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The elements of statistical learning: data mining, inference, and prediction. Springer Science & Business Media, 2009
[Hua+17]
Huang, X., et al.: Safety verification of deep neural networks. In: CAV, no. 1 (2017)
[JKO18]
Julian, K.D., Kochenderfer, M.J., Owen, M.P.: Deep Neural Network Compression for Aircraft Collision Avoidance Systems. In: CoRR abs/1810.04240 (2018)
[Kat+17]
Guy, K., et al.: Reluplex: an efficient SMT solver for verifying deep neural networks. In: CAV, no. 1 (2017)
[LeC98]
LeCun, Y.: The MNIST database of handwritten digits. http://​yann.​lecun.​com/​exdb/​mnist/​ (1998)
[MHN13]
Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: ICML (2013)
[PA19]
Pavithra, P., Zahra, R.A.: Abstraction based output range analysis for neural networks. In: NeurIPS (2019)
[Pap+16]
Papernot, N., et al.: The limitations of deep learning in adversarial settings. In: EuroS&P. IEEE (2016)
[Ped+11]
Pedregosa, F., et al.: Scikit-learn: machine Learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
[PT10]
Luca, P., Armando, T.: An abstraction-refinement approach to verification of artificial neural networks. In: CAV (2010)
[SB15]
Suraj, S., Venkatesh Babu, R.: Data-free parameter pruning for deep neural networks. In: BMVC (2015)
[Sin+19a]
Singh, G., et al.: An abstract domain for certifying neural networks. In: Proceedings ACM Program. Lang. vol. 3.POPL, 41:1–41:30 (2019)
[Sin+19b]
Singh, G., et al.: Boosting robustness certification of neural networks. In: ICLR (Poster) (2019)
[SVS19]
Su, J., Vasconcellos Vargas, D., Sakurai, K.: One pixel attack for fooling deep neural networks. IEEE Trans. Evol. Comput. 23(5), 828–841 (2019)
[YGK19]
Elboher, Y.Y., Gottschlich, J., Katz, G.: An abstraction-based framework for neural network verification. In: arXiv e-prints, arXiv:​1910.​14574 (2019)
[ZYZ18]
Zhong, G., Yao, H., Zhou, H.: Merging neurons for structure compression of deep networks. In: ICPR (2018)