Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Epileptic Seizure Detection Using CNN Read chapter Read first chapter

2021 | OriginalPaper | Chapter

DualPrune: A Dual Purpose Pruning of Convolutional Neural Networks for Resource-Constrained Devices

Authors : Tejalal Choudhary, Vipul Mishra, Anurag Goswami

Published in: Advanced Computing

Publisher: Springer Singapore

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Many successful applications of deep learning have been witnessed in various domains. However, the use of deep learning models in edge-devices is still limited. Deploying a large model onto small devices for real-time inference requires an adequate amount of resources. In the last couple of years, pruning has evolved as an important and widely used technique to reduce the inference cost and compress the storage-intensive deep learning models for small devices. In this paper, we proposed a novel dual-purpose pruning approach to accelerate the model performance and reduce the storage requirement of the model. The experiments on the CIFAR10 dataset with AlexNet and VGG16 models show that our proposed approach is effective and can be used to make the deployment of the trained model easier for edge-devices with marginal loss of accuracy. For the VGG16 experiment, our approach reduces parameters from 14.98M to 3.7M resulting in a 74.73% reduction in floating-point operations with only 0.8% loss in the accuracy.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter A Weighted Ensemble Approach to Real-Time Prediction of Suspended Particulate Matter
next chapter Incremental Ensemble of One Class Classifier for Data Streams with Concept Drift Adaption
Literature
1.
2.
Chen, S., Zhao, Q.: Shallowing deep networks: layer-wise pruning based on feature representations. IEEE Trans. Pattern Aanal. Mach. Intell. 41, 3048–3056 (2018) CrossRef
3.
Cheng, J., Wu, J., Leng, C., Wang, Y., Hu, Q.: Quantized CNN: a unified approach to accelerate and compress convolutional networks. IEEE Trans. Neural Netw. Learn. Syst. 29, 4730–4743 (2017)CrossRef
4.
Choudhary, T., Mishra, V., Goswami, A., Sarangapani, J.: A comprehensive survey on model compression and acceleration. Artif. Intell. Rev. 1–43 (2020)
5.
Courbariaux, M., Bengio, Y., David, J.P.: Binaryconnect: training deep neural networks with binary weights during propagations. In: Advances in Neural Information Processing Systems, pp. 3123–3131 (2015)
6.
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
7.
Denil, M., Shakibi, B., Dinh, L., De Freitas, N., et al.: Predicting parameters in deep learning. In: Advances in Neural Information Processing Systems, pp. 2148–2156 (2013)
8.
Denton, E.L., Zaremba, W., Bruna, J., LeCun, Y., Fergus, R.: Exploiting linear structure within convolutional networks for efficient evaluation. In: Advances in Neural Information Processing Systems, pp. 1269–1277 (2014)
9.
Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. In: Advances in Neural Information Processing Systems, pp. 1135–1143 (2015)
10.
Hassibi, B., Stork, D.G.: Second order derivatives for network pruning: Optimal brain surgeon. In: Advances in Neural Information Processing Systems, pp. 164–171 (1993)
11.
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
12.
Hinton, G.E., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. CoRR abs/1503.02531 (2015)
13.
Horowitz, M.: 1.1 computing’s energy problem (and what we can do about it). In: 2014 IEEE International Solid-State Circuits Conference Digest of Technical Papers (ISSCC), pp. 10–14. IEEE (2014)
14.
Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., Bengio, Y.: Quantized neural networks: training neural networks with low precision weights and activations. J. Mach. Learn. Rese. 18(1), 6869–6898 (2017)MathSciNetMATH
15.
16.
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images. Technical report, Citeseer (2009)
17.
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
18.
Lan, X., Zhu, X., Gong, S.: Knowledge distillation by on-the-fly native ensemble. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 7528–7538. Curran Associates Inc. (2018)
19.
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRef
20.
LeCun, Y., Denker, J.S., Solla, S.A.: Optimal brain damage. In: Advances in Neural Information Processing Systems, pp. 598–605 (1990)
21.
Li, F., Liu, B.: Ternary weight networks. In: 30th Conference on Neural Information Processing Systems (NIPS), Barcelona, Spain (2016)
22.
Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. Published as a conference paper at ICLR (2017)
23.
Lin, Z., Courbariaux, M., Memisevic, R., Bengio, Y.: Neural networks with few multiplications. Published as a conference paper at ICLR (2016)
24.
Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., Zhang, C.: Learning efficient convolutional networks through network slimming. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2736–2744 (2017)
25.
Luo, J.H., Zhang, H., Zhou, H.Y., Xie, C.W., Wu, J., Lin, W.: Thinet: Pruning CNN filters for a thinner net. IEEE Trans. Pattern Anal. Mach. Intel. 41, 2525–2538 (2018)CrossRef
26.
Paszke, A., et al.: Automatic differentiation in pytorch (2017)
27.
Shen, J., Vesdapunt, N., Boddeti, V.N., Kitani, K.M.: In teacher we trust: Learning compressed models for pedestrian detection. arXiv preprint arXiv:​1612.​00478 (2016)
28.
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Published as a Conference Paper at ICLR (2015)
29.
30.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)MathSciNetMATH
31.
Swaminathan, S., Garg, D., Kannan, R., Andres, F.: Sparse low rank factorization for deep neural network compression. Neurocomputing (2020)
32.
Yang, T.J., Chen, Y.H., Sze, V.: Designing energy-efficient convolutional neural networks using energy-aware pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 5687–5695 (2017)
33.
Zhang, X., Zou, J., He, K., Sun, J.: Accelerating very deep convolutional networks for classification and detection. IEEE Trans. Pattern Anal. Mach. Intell. 38(10), 1943–1955 (2016)CrossRef
34.
Zhao, C., Ni, B., Zhang, J., Zhao, Q., Zhang, W., Tian, Q.: Variational convolutional neural network pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2780–2789 (2019)
Metadata
Title
DualPrune: A Dual Purpose Pruning of Convolutional Neural Networks for Resource-Constrained Devices
Authors
Tejalal Choudhary
Vipul Mishra
Anurag Goswami
Copyright Year
2021
Publisher
Springer Singapore
Book
Advanced Computing

Print ISBN: 978-981-16-0400-3

Electronic ISBN: 978-981-16-0401-0

Copyright Year: 2021

DOI
https://doi.org/10.1007/978-981-16-0401-0_30

Premium Partner

    Image Credits