DRP:Discrete Rank Pruning for Neural Network

Although deep neural networks (DNNs) have achieved excellent performance in computer vision applications in recent years, it is still challenging to deploy them on resource-limited devices such as mobile phones. To solve this problem, we propose a novel filter pruning method for neural network named Discrete Rank Pruning (DRP). Moreover, many methods apply sparse regularization on the filter weights of the convolution layers to reduce the degradation of performance after pruning. We analyze these methods and find that it is necessary to consider the influence of the bias term. Based on these, we propose a novel sparse method named Consideration Bias Sparsity (CBS). Extensive experiments on MNIST, CIFAR-10 and CIFAR-100 datasets with LeNet-5, VGGNet-16, ResNet-56, GoogLeNet and DenseNet-40 demonstrate the effectiveness of CBS and DRP. For LeNet-5, CBS achieves 1.87% increase in accuracy than sparse regularization on MNIST. For VGGNet-16, DRP achieves 66.6% reduction in FLOPs by removing 83.3% parameters with only 0.36% decrease in accuracy on CIFAR-10. For ResNet-56, DRP achieves 47.45% reduction in FLOPs by removing 42.35% parameters with only 0.82% decrease in accuracy on CIFAR-100.