Classification of orbital tumors using convolutional neural networks

We used two datasets in our experiment. Both types of images of orbital tumors were collected from Ain Shams University Hospital, Ophthalmology Department. The normal images were collected manually from different articles from a public website [18]. They consist of different MRI images: T1-weighted and T2-weighted images. The total size of T1-weighted images is 133 MRI images: 20 normal images and 113 orbit tumor images. The total size of T2-weighted images is 109 MRI images: 24 normal images and 85 orbit tumor images. The tumor datasets consist of 30 patients that have different orbital tumor types. The tumor T1-weighted image dimension is (800 × 600) pixel resolution, and the normal image is (509 × 400) pixel resolution. The tumor T2-weighted image dimension is (824 × 800) pixel resolution, and the normal image is (516 × 471) pixel resolution. Figure 3 shows the different MRI image samples used in the experiment.

In our work, datasets had small samples that were not enough to be used in the training phase. In order to prevent overfitting in the experiment, one of the resampling techniques was applied. First, oversampling technique was implemented for each type of MRI image (normal and tumor) to upgrade the number of images that were used in the classification. Many image processing techniques were applied including Gaussian blur filter, Median filter, unsharp mask filter, sharpening filters, edge enhancement, brightness filter and contrast filter.

Gaussian blur filter is a low pass filter used for reducing noise (high-frequency components) and blurring regions of an image and it is calculated by Eq. 1 [19].where σ is the standard deviation. In the context of filtering, the standard deviation σ is a parameter, which determines the width of the filter.

Median filtering is a nonlinear method used to remove noise from images. It is widely used as it is very effective at removing noise while preserving edges. It is particularly effective at removing ‘salt and pepper’ type noise. The median filter works by moving through the image pixel by pixel, replacing each value with the median value of neighboring pixels. The pattern of neighbors is called the "window," which slides pixel by pixel, over the entire image. [19]. The unsharp mask filter is an extremely versatile sharpening tool that improves the definition of fine detail by removing low-frequency spatial information from the original image. It involves the subtraction of an unsharp mask from the specimen image. An unsharp mask is simply a blurred image produced by spatially filtering the specimen image with a Gaussian low-pass filter [20]. The sharpening filters are a type of image processing that enhances the contrast between neighboring pixels, making the edges and details more visible and defined. Sharpening filters can also reduce the effects of noise, blur, or compression artifacts that degrade the quality of digital images. Edge enhancement is an image processing filter that enhances the edge contrast of an image or video in an attempt to improve its acutance (apparent sharpness). The filter works by identifying sharp edge boundaries in the image, such as the edge between a subject and a background of a contrasting color and increasing the image contrast in the area immediately around the edge. This has the effect of creating subtle bright and dark highlights on either side of any edges in the image, called overshoot and undershoot, leading the edge to look more defined when viewed from a typical viewing distance [21]. Brightness is a relative term defined as the intensity of a pixel relative to another pixel. To increase the brightness, we need to increase the intensity of each pixel by a constant and similarly to darken the image we need to decrease the intensity of every pixel of the image [22]. Contrast can simply be explained as the difference between maximum and minimum pixel intensity in an image. To change the contrast of an image we just need to change the value of the max and min intensity pixels [23].

For normal type, Gaussian blur filter, Median filter, Unsharp mask filter with radius size (0.5, 1, 2 and 3), sharpness filter with factor (0.5, 1, 1.5, 2, 3 and 4), EDGE Enhance Filter, EDGE Enhance More Filter, Brightness Filter with factor (0.5) and Contrast Filter with factor (0.5) were applied to images. For tumor type, median filter, sharpness filter with factors (0.5, 1, 3 and 4) and Unsharp Mask filter with radius size (1.5) were executed on images. Figure 4 shows samples for MRI T1-weighted images after applying the image filter. Figure 5 shows samples for MRI T2-weighted images after applying the image filter.

The total number of T1-weighted images is 1404 MRI images: 702 are normal and 702 are tumor images, and the total number of T2-weighted images is 1560 MRI images: 780 are normal and 780 are tumor images. Tables 1 and 2 show the total number of MRI images after applying the image processing techniques.

Second, the datasets in our experiment were split into training, validation and testing sets with a ratio of 80% and 20% rule; 10% is validation and 10% is testing. The 80–20 split ratio is prevalently used in deep learning and is used frequently in medical images. This 80–20 split technique is known as "image-level approach". In this split, each patient image is included in either the training or validation set, but not in both and this is known as "patient-level method" [24].

Many medical imaging collections in clinical settings suffer from imbalanced data, and it is rare to find a large amount of data for a specific clinical case. The skewed data distribution is an essential issue that occurs frequently in medical image classification problems. One of the methods to overcome the insufficient amount of data utilized in the training phase is data augmentation [25]. It is a method of increasing the dataset size by implementing various modifications such as rescaling, resizing and rotation of current data in the runtime to generate new samples while keeping the same label. Augmenting images increases the overall amount of images available to the model, permitting it to learn more effectively [26]. Data augmentation has been deemed as a sort of dataset regularization since it reduces over-fitting and improves overall performance by enriching the training dataset itself [27]. Moreover, data augmentation is used to solve the issue of imbalanced classification by oversampling the minority class to enlarge the datasets to make the model perform better on the training data [28].

In our experiment, the dataset has small samples and we used data augmentation to increase the amount of data. To prevent over-fitting in the training phase, we applied some geometric transformation techniques to the MRI images to enhance the training phase. First, the images were resized which changed the width and height to 224 × 224 pixels. Second, images were rescaled which transformed the pixel range from [0, 255] to [0, 1]. Third, images were rotated at 40° degrees. Fourth, the width and height of images had shifted by 20%. Fifth, images were sheared by 20%. Sixth, images were zoomed in by 20%. Finally, images were flipped horizontally which reflect the images around the central horizontal axis.

After applying image augmentation, the total number of T1-weighted images is 11,232 MRI images and the total number of T2-weighted images is 12,480 MRI images. Table 3 shows the total number of MRI images after applying image augmentation techniques.

Our CNN model is split into fifteen sequence levels. It was trained with 80 epochs and 64 batch sizes. The optimizer type of CNN is ‘Adam,’ the loss function is binary cross-entropy, and the learning rate is 0.01. Figure 6 illustrates the CNN model architecture.

Layer 1 in our CNN architecture is the convolutional layer with the ReLu (Rectified Linear Unit) activation function. This layer obtains the pre-processed image as an input with a size n*n = 224*224. The convolutional kernel size (filter size) is f*f = 3*3, there is no padding (p), stride (s) is 1 and the number of filters (neurons) is 32. We get feature maps of size 32@222*222 after this convolution process, where 32 is the number of neurons and 222 is the output of the formula ((n + 2p−f)/s) + 1 = ((224 + 2*0–3)/1) + 1 = 222. The output of this layer recognizes essential characteristics such as straight edges and corners.

Layer 2 is the subsampling layer which we use as the max pooling layer. The pooling size is 2*2, padding is 0 and stride is the default value so it is the same as pooling size 2. We get a feature map of size 32@111*111 after the max pooling operation, where 32 is the number of neurons and 111 is the output of the formula ((n + 2p−f)/s) + 1 = ((222 + 2*0–2)/2) + 1 = 111. This layer has no activation function.

Layer 3 is the convolutional layer where the ReLu activation function is applied. The convolutional filter size is 3*3, the padding is 0, the stride is 1, and the number of neurons is 64. After this convolution process, we get feature maps of size 64@109*109, where 64 is the number of neurons. The ReLu activation is applied in each feature map.

Layer 4 is the max pooling layer. The pooling size is 2*2, padding is 0, and stride is 2. We get a feature map of size 64@54*54 after the max pooling operation, where 64 is the number of neurons. This layer has no activation function.

Layer 5 is the convolutional layer where the ReLu activation function is applied. The convolutional filter size is 3*3, the padding is 0, the stride is 1, and the number of neurons is 128. After this convolution process, we get feature maps of size 128@52*52, where 128 is the number of neurons. The ReLu activation is applied to each feature map.

Layer 6 is the max pooling layer. The pooling size is 2*2, padding is 0 and stride is 2. We get feature map size 128@26*26 after the max pooling operation, where 128 is the number of neurons. This layer has no activation function.

Layer 7 is the convolutional layer where the ReLu activation function is applied. The convolutional filter size is 3*3, the padding is 0, the stride is 1, and the number of neurons is 64. After this convolution process, we get feature maps of size 64@24*24, where 64 is the number of neurons. The ReLu activation is applied to each feature map.

Layer 8 is the max pooling layer. The pooling size is 2*2, padding is 0, and stride is 2. We get a feature map of size 64@12*12 after the max pooling operation, where 64 is the number of neurons. This layer has no activation function.

Layer 9 is the convolutional layer where the ReLu activation function is applied. The convolutional filter size is 3*3, the padding is 0, the stride is 1, and the number of neurons is 32. After this convolution process, we get feature maps of size 32@10*10, where 32 is the number of neurons. The ReLu activation is applied to each feature map.

Layer 10 is the max pooling layer. The pooling size is 2*2, padding is 0, and stride is 2. We get a feature map of size 32@5*5 after the max pooling operation, where 32 is the number of neurons. This layer has no activation function.

Layer 11 is a flattened layer to reshape the previous layer’s output and produces a one-dimensional vector to be used as an input to a fully connected layer.

Layer 12 is a dropout layer, a mask that eliminates some neurons' contributions to the next layer. It is one of the regularization techniques to prevent overfitting during the training phase.

Layer 13 is a fully connected layer. It receives the input from the flattened layer and outputs a one-dimensional vector of size 128. Each element in the vector receives the ReLu activation function.

Layer 14 is a fully connected layer. It receives the input from the previous layer and outputs a one-dimensional vector of size 64. Each element in the vector receives the ReLu activation function.

Layer 15 is the last layer in our CNN architecture. It is another fully connected layer. It calculates the class scores for each image to which the class belongs, resulting in a binary class [0, 1] where the normal image is represented by 0 and the tumor image is represented by 1. For the final output, the sigmoid activation function is used.