Res-CapsNet: Residual Capsule Network for Data Classification

Capsule network is a new network structure which can encode the properties and spatial relations of image features. It overcomes the shortcomings of CNN that requiring large number of training samples and parameters, and information loss during the pooling process. Capsule network can only extract shallow features, which makes it perform poorly on complex datasets. In this paper, a new residual capsule network model (Res-CapsNet) is proposed by fusing capsule network, residual network and deconvolution. The Res-CapsNet extracts deep features and sends them to capsule module using the dense connections of residual, which effectively strengthens the feature transfer and feature utilization. Capsule module converts scalar neurons to vector neurons through the main capsule layer, and uses dynamic routing algorithm to selectively activate the high-level capsule in the main capsule layer and the digital capsule layer, and obtains the recognition results. Deconvolution reconstruction module is the last part of Res-CapsNet, responsible for reconstructing recognition results by using 4-layer deconvolution. Res-CapsNet utilizes beta-mish activation function to reduce the "death" of neurons caused by ReLU, thus activating more neurons to further improve the classification accuracy. The experimental results show that Res-CapsNet has better performance on datasets, such as SVHN, FASH-MNIST and CIFAR-10. Compared with the baseline model CapsNet, the model parameters of Res-CapsNet on dataset CIFAR-10 are reduced by 65.73%, while the classification accuracy is significantly improved by 33.66%.