ResCaps: an improved capsule network and its application in ultrasonic image classification of thyroid papillary carcinoma

Thyroid cancer is the most common thyroid malignancy, which accounts for about \(1\%\) of systemic malignancy. As a type of thyroid cancer, thyroid papillary carcinoma accounts for the largest proportion of thyroid cancer [1], and can occur at any age. Recently, more and more people suffer from such diseases. Thus, it is especially important to improve the accuracy of diagnosis of papillary thyroid carcinoma. Ordinarily, the ultrasonic images of thyroid papillary carcinoma have low resolution and poor definition. Moreover, the thyroid gland has the complex tissue structure and many interfering factors in the ultrasonic images. Thus, it is difficult to classify the features of the lesions in the ultrasonic images of thyroid cancer.

In recent years, the classification of medical images is usually achieved by convolutional neural network (C-NN) [2]. Sabour et al. used a CNN to distinguish the brain of Alzheimer’s patients with normal and healthy brains [3]. The famous LeNet-5 network model was used to classify functional magnetic resonance imaging data of Alzheimer’s patients from the normal control group, which has the accuracy rate of \(96.85\%\). [4] proposed a method to improve the Faster RCNN scheme for detecting speed and to promote the efficiency of classification accuracy. This scheme used the convolution of fourth and fifth layer to take L2 regularization processing. Then, the two layers of realizing information sharing with multi-scale were connected as the input port. Finally, the accuracy and speed of the network model recognition effect is improved, respectively. Li et al. added a spatial constraint layer to CNN, which enabled the detector to extract the features around the cancer area [5]. In addition, the more obscure or smaller cancer areas could be detected by the detector, because the shallow and deep layers of CNN are connected. However, the detector contains a large number of parameters, which is not easy to transplant into the actual terminal.

In order to overcome the shortcoming that CNN cannot detect tissue overlap features effectively, Sarraf et al. proposed the capsule network model [6]. Obviously, the capsule network model increased the degree of correlation between the local and the whole. Moreover, this model represented the spatial characteristics of the object by the vector contained more information. The capsule network model is tested by the handwritten data set (MNIST) [7] and its accuracy is \(99.64\%\). To improve these problems, such as large amount of capsule layer data and slow training speed, Geoffrey et al. proposed a matrix capsule network model [8]. The matrix capsule network model used Gaussian mixture model to complete the routing layer clustering process, which reduces the amount of data in the capsule layer. Moreover, this model is changed from a vector form to a matrix and a scalar form, where the activation value determines the probability of being selected between the upper and lower capsule layers. Thus, the experimental training speed is accelerated. Mobiny et al. proposed a rapid capsule network for the classification of lung cancer images pretreatment [9]. The fast capsule network used the same pixel value to equal the same routing correlation coefficient, which reduces the complexity of routing layer operation. Moreover, its network training speed was three times that of the capsule network. In addition, the deconvolution layer was used to replace the full connection layer of the capsule network in the rapid capsule network, which prevented overfitting and ultimately improved the classification accuracy. In summary, the capsule network is significantly better than the traditional convolutional neural network in terms of accuracy of overlapping object recognition and it contains rich information [10, 11]. Therefore, the capsule network model is better than the traditional CNN in terms of the classification task.

With the continuous development of capsule network model in the field of medical imaging, this model has been fully applied in the accurate reconstruction of functional nuclear magnetic resonance [12]. A new model based on capsule network, named SegCap, was applied to the segmentation of low-dose CT scan images, which accuracy is significantly improved [13]. Recently, the capsule network model has been continuously optimized and applied. For example, CapsGan used capsule network as a discriminator of GAN, which obtained better visualization effect than the traditional GAN network based on convolution [14, 15]. Moreover, DeepCaps network [16] fused the dynamic routing algorithm of capsules on the basis of deep convolutional network, which reduced the number of parameters of the original capsule network by \(68\%\) and achieving the world’s leading level in CIFAR10 [17] and Fashion-MNIST open data sets [18]. Thus, the fusion of the discriminator network, the capsule network and the deep convolutional network is also a development trend in the future.

Motivated by the above research methods, the ultrasonic image classification task of thyroid papillary carcinoma is proposed in this paper. The main contributions of this paper are reported as follows.

The model structure of the capsule network is shown in (a) from Fig. 1. The capsule network is consisted of the ultrasonic image input layer, convolutional neural network layer, PrimaryCaps initial capsule layer, DigitCaps and the final output layer, which are introduced as follows:

Input layer: it is used to input the ultrasonic image of pre-treated papillary carcinoma of thyroid.

PrimaryCaps initial capsule layer: the primarily low-level features can be stored as a vector. In the initial capsule layer, the spatial dimension is transformed, and the expression of the vector is obtained to prepare for the input of the next layer.

Digital capsule layer: this layer is connected with the initial capsule layer by the form of the vector to vector. Moreover, the output of this layer is calculated using a dynamic routing algorithm.

Each capsule unit of the capsule layer is represented by a vector, where a capsule unit contains the attitude parameters of the object and the probability of belonging to this class. In the traditional convolutional network, the pooling layer is a shortcoming. In the process of dimensionality reduction, the layer loses some important feature information, which leads to reducing identification accuracy. The capsule network is used in the network dynamic routing method, which is an effective way to solve the above problem [7]. For dynamic routing method, the correlation of fluctuation capsules layer can be updated by judging the weight value. Interestingly, some trade-offs between DigitCaps layer capsule unit and the proportion of the learning process are required. If the predicted results is closed to the real value, the associated values and capsule unit of DigitCaps layer are promoted. If the predicted results and the real value are far off, the correlation value and certain capsule unit of DigitCaps layer are declined. The specific calculation process of dynamic routing is given as follows:where \({u_i}\) is the output of the ith capsule layer, \({u_{j|i}}\) means the prediction vector output of the jth layer obtained by calculating the ith capsule layer, \({W_{ij}}\) is the weight value that is used for learning and back propagation:where \(c_{ij}\) is the associated value of adjacent capsule layers, \(b_{ij}\) is the probability that the ith capsule layer is selected by the jth capsule layer. The initial value of \(b_{ij}\) is set to 0, when the routing layer starts to execute:where \({s_j}\) is the input vector of the jth capsule layer:where \({v_j}\) is the output of the jth capsule layer; the output value \(\frac{{{s_j}}}{{\left\| {{s_j}} \right\| }}\) is guaranteed to be in the interval [0, 1], which does not exceed the expected range value. (4) is the remainder of the non-linear activation function:where the input \({\widehat{u}_{j\left| \mathrm{{i}} \right. }}\) and output \({v_j}\) are updated by the inner product between vectors. The value of \({b_{ij}}\) is a parameter, which adjusts the degree of correlation between the capsule layers:In (6), \({L_c}\) is the loss function of the classified network modules. c is the category of classification. \({T_{c}}\) is an indicator function of the classification. \({T_{c}}=1\) when c is existence, while \({T_{c}}=0\) when c does not exist. \({m^+}\) means the upper edge, i.e., the punishment false negative; \({m^-}\) means the lower margin, i.e., false positives are penalized. \(\lambda \) denotes the proportional coefficient, which is used to adjust the proportion between \({m^+}\) and \({m^-}\). \({m^+}\), \({m^-}\) and \(\lambda \) are hyper parameters that have been set before the capsule network learning.

The convolution layer of the capsule network has a low ability to process feature information, which is not enough to provide more detailed advanced feature information for the initial capsule layer. To solve such problem, a network model with better performance is proposed in this paper, which is named ResCaps network model.

The structure of the new ResCaps network is shown in (b) from Fig. 1. Based on CapsNet network model, residual modules are fused. The size of the input image is not changed after the ultrasonic image passes through the residual module. The information processed by the residual module provides advanced features for the initial capsule layer better.

The principle of a single residual module is shown in Fig. 2. The residual module adds a quick connection between the input and output of the network layer, which is the identity map:where X represents the input of the residual block, H(X) is the output of the residual block, and F(X) is the residual:which is the deformation of (7). The residual module represents the network, which fits the residual F(X) between the input and output. The residual module can deepen the network model and improve the ability of abstract expression of features in the ResCaps network module.

The specific composition of the residual module is shown in Fig. 3. The residual module is composed of modules with a convolution kernel size of \(1\times 1\), convolution kernel size of \(3\times 3\) and channel number of 32, respectively. Here, the network model name is given by the number of residual modules, where one residual block is named ResCaps1, two residual blocks are named ResCaps2 and so on.

As shown in (c) from Fig. 1, the CNNCaps network model corresponds to the ResCaps network model structure. However, their difference is the module before the PrimaryCaps. The CNN modules are consistent with the convolution parameters in the residual module. It is missing that the shortcut connection in the residual module. The CNNCaps network model can well reflect the advantages and disadvantages of the residual module and the CapsNet network model.

The CNNCaps network model is named in the same way as the ResCaps network, and the name is determined by the corresponding CNN module. For instance, the model contains a CNN module, which is named CNNCaps1. The model contains two CNN modules, which is named CNNCaps2.

In this paper, the pretreatment scheme based on the bilinear interpolation method is proposed to obtain the ultrasonic images of thyroid papillary carcinoma with a specified size, which satisfies the input of the capsule network. Furthermore, the ResCaps network model is designed, which is applicable to the classification task of thyroid papillary carcinoma. On the premise that the network model does not affect the training speed of CapsNet network model, the accuracy of the classification task is improved and reaches the optimal accuracy of \(81.06\%\). However, the current ResCaps network model still has room for upgrading. Thus, the future research direction is how to modify the network model structure to further improve the accuracy of model classification.