Skip to main content

Tipp

Weitere Kapitel dieses Buchs durch Wischen aufrufen

2020 | OriginalPaper | Buchkapitel

DeepAbstract: Neural Network Abstraction for Accelerating Verification

verfasst von : Pranav Ashok, Vahid Hashemi, Jan Křetínský, Stefanie Mohr

Erschienen in: Automated Technology for Verification and Analysis

Verlag: Springer International Publishing

share
TEILEN

Abstract

While abstraction is a classic tool of verification to scale it up, it is not used very often for verifying neural networks. However, it can help with the still open task of scaling existing algorithms to state-of-the-art network architectures. We introduce an abstraction framework applicable to fully-connected feed-forward neural networks based on clustering of neurons that behave similarly on some inputs. For the particular case of ReLU, we additionally provide error bounds incurred by the abstraction. We show how the abstraction reduces the size of the network, while preserving its accuracy, and how verification results on the abstract network can be transferred back to the original network.
Fußnoten
1
Naturally, the parameter \(\alpha \) has to be less than or equal to the accuracy of \(D\).
 
Literatur
[AM18]
Zurück zum Zitat Akhtar, N., Mian, A.: Threat of adversarial attacks on deep learning in computer vision: a survey. In: IEEE Access, vol. 6, pp. 14410–14430 (2018) Akhtar, N., Mian, A.: Threat of adversarial attacks on deep learning in computer vision: a survey. In: IEEE Access, vol. 6, pp. 14410–14430 (2018)
[Ash+20]
[Bis06]
Zurück zum Zitat Christopher M Bishop. Pattern recognition and machine learning. springer, 2006 Christopher M Bishop. Pattern recognition and machine learning. springer, 2006
[CGL94]
Zurück zum Zitat Clarke, E.M., Grumberg, O., Long, D.E.: Model checking and abstraction. ACM Trans. Program. Lang. Syst. 16(5), 1512–1542 (1994) Clarke, E.M., Grumberg, O., Long, D.E.: Model checking and abstraction. ACM Trans. Program. Lang. Syst. 16(5), 1512–1542 (1994)
[Che+17a]
Zurück zum Zitat Chen, X., et al.: Multi-view 3d object detection network for autonomous driving. In: CVPR (2017) Chen, X., et al.: Multi-view 3d object detection network for autonomous driving. In: CVPR (2017)
[Che+17b]
Zurück zum Zitat Yu, C., et al.: A Survey of Model Compression and Acceleration for Deep Neural Networks. In: CoRR abs/1710.09282 (2017) Yu, C., et al.: A Survey of Model Compression and Acceleration for Deep Neural Networks. In: CoRR abs/1710.09282 (2017)
[Cla+00]
Zurück zum Zitat Clarke, E.M., et al.: Counterexample-guided abstraction refinement. In: CAV (2000) Clarke, E.M., et al.: Counterexample-guided abstraction refinement. In: CAV (2000)
[CNR17]
Zurück zum Zitat Chih-Hong, C., Nührenberg, G., Ruess, H.: Maximum resilience of artificial neural networks. In: ATVA (2017) Chih-Hong, C., Nührenberg, G., Ruess, H.: Maximum resilience of artificial neural networks. In: ATVA (2017)
[Den+20]
Zurück zum Zitat Lei, D., et al.: Model compression and hardware acceleration for neural networks: a comprehensive survey. In: Proceedings of the IEEE 108(4), 485–532 (2020) Lei, D., et al.: Model compression and hardware acceleration for neural networks: a comprehensive survey. In: Proceedings of the IEEE 108(4), 485–532 (2020)
[Don+18]
Zurück zum Zitat Dong, Y., et al.: Boosting adversarial attacks with momentum. In: CVPR (2018) Dong, Y., et al.: Boosting adversarial attacks with momentum. In: CVPR (2018)
[Dvi+18]
Zurück zum Zitat Krishnamurthy, D., et al.: A dual approach to scalable verification of deep networks. In: UAI (2018) Krishnamurthy, D., et al.: A dual approach to scalable verification of deep networks. In: UAI (2018)
[Ehl17]
Zurück zum Zitat Rüdiger, E.: Formal verification of piece-wise linear feed- forward neural networks. In: ATVA (2017) Rüdiger, E.: Formal verification of piece-wise linear feed- forward neural networks. In: ATVA (2017)
[Geh+18]
Zurück zum Zitat Timon, G., et al.: Ai2: safety and robustness certification of neural networks with abstract interpretation. In: 2018 IEEE Symposium on Security and Privacy (SP) (2018) Timon, G., et al.: Ai2: safety and robustness certification of neural networks with abstract interpretation. In: 2018 IEEE Symposium on Security and Privacy (SP) (2018)
[HMD16]
Zurück zum Zitat Han, S., Mao, H., Dally, W.J.: Deep compression: compressing deep neural network with pruning, trained quantization and huffman coding. In: ICLR (2016) Han, S., Mao, H., Dally, W.J.: Deep compression: compressing deep neural network with pruning, trained quantization and huffman coding. In: ICLR (2016)
[HTF09]
Zurück zum Zitat Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The elements of statistical learning: data mining, inference, and prediction. Springer Science & Business Media, 2009 Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The elements of statistical learning: data mining, inference, and prediction. Springer Science & Business Media, 2009
[Hua+17]
Zurück zum Zitat Huang, X., et al.: Safety verification of deep neural networks. In: CAV, no. 1 (2017) Huang, X., et al.: Safety verification of deep neural networks. In: CAV, no. 1 (2017)
[JKO18]
Zurück zum Zitat Julian, K.D., Kochenderfer, M.J., Owen, M.P.: Deep Neural Network Compression for Aircraft Collision Avoidance Systems. In: CoRR abs/1810.04240 (2018) Julian, K.D., Kochenderfer, M.J., Owen, M.P.: Deep Neural Network Compression for Aircraft Collision Avoidance Systems. In: CoRR abs/1810.04240 (2018)
[Kat+17]
Zurück zum Zitat Guy, K., et al.: Reluplex: an efficient SMT solver for verifying deep neural networks. In: CAV, no. 1 (2017) Guy, K., et al.: Reluplex: an efficient SMT solver for verifying deep neural networks. In: CAV, no. 1 (2017)
[MHN13]
Zurück zum Zitat Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: ICML (2013) Maas, A.L., Hannun, A.Y., Ng, A.Y.: Rectifier nonlinearities improve neural network acoustic models. In: ICML (2013)
[PA19]
Zurück zum Zitat Pavithra, P., Zahra, R.A.: Abstraction based output range analysis for neural networks. In: NeurIPS (2019) Pavithra, P., Zahra, R.A.: Abstraction based output range analysis for neural networks. In: NeurIPS (2019)
[Pap+16]
Zurück zum Zitat Papernot, N., et al.: The limitations of deep learning in adversarial settings. In: EuroS&P. IEEE (2016) Papernot, N., et al.: The limitations of deep learning in adversarial settings. In: EuroS&P. IEEE (2016)
[Ped+11]
Zurück zum Zitat Pedregosa, F., et al.: Scikit-learn: machine Learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011) Pedregosa, F., et al.: Scikit-learn: machine Learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
[PT10]
Zurück zum Zitat Luca, P., Armando, T.: An abstraction-refinement approach to verification of artificial neural networks. In: CAV (2010) Luca, P., Armando, T.: An abstraction-refinement approach to verification of artificial neural networks. In: CAV (2010)
[SB15]
Zurück zum Zitat Suraj, S., Venkatesh Babu, R.: Data-free parameter pruning for deep neural networks. In: BMVC (2015) Suraj, S., Venkatesh Babu, R.: Data-free parameter pruning for deep neural networks. In: BMVC (2015)
[Sin+19a]
Zurück zum Zitat Singh, G., et al.: An abstract domain for certifying neural networks. In: Proceedings ACM Program. Lang. vol. 3.POPL, 41:1–41:30 (2019) Singh, G., et al.: An abstract domain for certifying neural networks. In: Proceedings ACM Program. Lang. vol. 3.POPL, 41:1–41:30 (2019)
[Sin+19b]
Zurück zum Zitat Singh, G., et al.: Boosting robustness certification of neural networks. In: ICLR (Poster) (2019) Singh, G., et al.: Boosting robustness certification of neural networks. In: ICLR (Poster) (2019)
[SVS19]
Zurück zum Zitat Su, J., Vasconcellos Vargas, D., Sakurai, K.: One pixel attack for fooling deep neural networks. IEEE Trans. Evol. Comput. 23(5), 828–841 (2019) Su, J., Vasconcellos Vargas, D., Sakurai, K.: One pixel attack for fooling deep neural networks. IEEE Trans. Evol. Comput. 23(5), 828–841 (2019)
[YGK19]
Zurück zum Zitat Elboher, Y.Y., Gottschlich, J., Katz, G.: An abstraction-based framework for neural network verification. In: arXiv e-prints, arXiv:​1910.​14574 (2019) Elboher, Y.Y., Gottschlich, J., Katz, G.: An abstraction-based framework for neural network verification. In: arXiv e-prints, arXiv:​1910.​14574 (2019)
[ZYZ18]
Zurück zum Zitat Zhong, G., Yao, H., Zhou, H.: Merging neurons for structure compression of deep networks. In: ICPR (2018) Zhong, G., Yao, H., Zhou, H.: Merging neurons for structure compression of deep networks. In: ICPR (2018)
Metadaten
Titel
DeepAbstract: Neural Network Abstraction for Accelerating Verification
verfasst von
Pranav Ashok
Vahid Hashemi
Jan Křetínský
Stefanie Mohr
Copyright-Jahr
2020
DOI
https://doi.org/10.1007/978-3-030-59152-6_5

Premium Partner