1 Introduction
-
New threshold determination approach for SWP: The research proposes a new method for determining which weights in a neural network can be pruned without sacrificing accuracy. Our pruned VGG16 achieves results comparable to the existing model, with a fourfold reduction in parameters and only a 0.4% decrease in accuracy.
-
Stable theoretical basis: The proposed method is based on sound theoretical principles, making it more trustworthy and easier to understand and apply.
-
Deployment of different deep layers on edge devices: The effectiveness of the proposed approach is tested on different neural network architectures (VGG11, VGG13, VGG16, and ResNet56) and evaluated on edge devices with limited computational resources.
2 Related work
3 The proposed method
3.1 Stripe wise convolution
3.2 Theoretical analysis
3.3 Method description
3.4 Computational complexity
4 Experiments
4.1 Experiments on CIFAR-10
4.1.1 Implementation details
4.2 Experiments on bearings dataset
4.2.1 Symmetrized dot pattern
4.2.2 Data categories and preprocess
Fault diameter | Inner race | Ball | Outer race position relative to load zone | ||
---|---|---|---|---|---|
centered @ 6 | orthogonal @ 3 | opposite @ 12 | |||
0.007” | IR007 | B007 | OR007 @ 6 | OR007 @ 3 | OR007 @ 12 |
0.014” | IR014 | B014 | – | – | – |
0.021” | IR021 | B021 | OR021 @ 6 | OR021 @ 3 | OR021 @ 12 |
-
Training data and testing data are both from vibration signals under the same working load.
-
e.g., Training data and testing data are both from vibration signals under working load of 0 hp.
-
-
Training data come from vibration signals under a certain working load while testing data are from different working loads.
-
e.g., Training data come from vibration signals under working load 0 hp while testing data are from working load of 3 hp.
-
5 Results
5.1 Results on on CIFAR-10
5.1.1 Comparing with the original SWP
Backbone | Metrics | Params | FLOPS | Accuracy |
---|---|---|---|---|
VGG16 | Baseline | 14.76M | 627.37 M | 93.76% |
Original (\(\alpha =1\)e−5) | 3.62M | 350.28 M | 93.46% | |
Original (\(\alpha =5\)e−5) | could not converge | |||
Ours (\(\alpha =1\)e−5, \(T=0.0001\)) | 4.63 M | 385.49 M | 93.43% | |
Ours (\(\alpha =5\)e−5, \(T=0.005\)) | 0.84 M | 126.03 M | 93.06% | |
ResNet56 | Baseline | 0.87M | 251.50 M | 93.11% |
Original (\(\alpha =1\)e−5) | 0.60M | 150.63 M | 93.41% | |
Ours (\(\alpha =5\)e−5, \(T=0.001\)) | 0.23 M | 60.76 M | 92.96% |
5.1.2 Ablation study
\(\alpha\) | 1e−5 | 5e−5 | |||||
---|---|---|---|---|---|---|---|
T | 0.0001 | 0.001 | 0.01 | 0.0001 | 0.0005 | 0.001 | 0.005 |
Params (M) | 4.63 | 4,17 | 2.89 | 0.78 | 0.80 | 0.79 | 0.84 |
FLOPS (M) | 385.49 | 327.17 | 200.09 | 130.96 | 135.56 | 134.68 | 126.03 |
Accuracy (%) | 93.43 | 93.28 | 92.99 | 92.79 | 92.86 | 92.96 | 93.06 |
5.1.3 Edge device performance
5.2 Results on bearings dataset
5.2.1 Comparison of the original model and the pruned model
Sub-dataset | Accuracy |
---|---|
(a) Original model | |
D00 | 0.94385 |
D01 | 0.99277 |
D02 | 0.92292 |
D03 | 0.97400 |
D10 | 0.98923 |
D11 | 0.96615 |
D12 | 0.97908 |
D13 | 0.98846 |
D20 | 0.98031 |
D21 | 0.98308 |
D22 | 0.97077 |
D23 | 0.98892 |
D30 | 0.98600 |
D31 | 0.98092 |
D32 | 0.98877 |
D33 | 0.96462 |
Pruned model | |
D00 | 0.97846 |
D01 | 0.99323 |
D02 | 0.98800 |
D03 | 0.97246 |
D10 | 0.98785 |
D11 | 0.95077 |
D12 | 0.98554 |
D13 | 0.99031 |
D20 | 0.97477 |
D21 | 0.98600 |
D22 | 0.97000 |
D23 | 0.99031 |
D30 | 0.98923 |
D31 | 0.98062 |
D32 | 0.97754 |
D33 | 0.97615 |
Backbone | Metrics | Params | FLOPS | Accuracy |
---|---|---|---|---|
VGG16 | Baseline | 14.76 M | 627.37 M | 97.40% |
Original (\(\alpha =1e-5\)) | 3.32 M | 341.65 M | 97.35% | |
Ours (\(\alpha =1e-5\), \(T=0.0001\)) | 4.43 M | 375.68 M | 97.43% |