Introduction
Literature review
Mathematical model
Materials and methods
Convolutional neural networks
Transfer learning
Model | Year | Source | Top-1 accuracy (ImagNet) (%) |
---|---|---|---|
AlexNet | 2012 [45] | BVLC | 57.1 |
VGG16 | 2014 [46] | Oxford | 70.5 |
VGG19 | 2014 [46] | Oxford | 71.3 |
Inception V1 | 2015 [47] | Google | 67.9 |
SqueezeNet | 2016 [48] | DeepScale | 59.5 |
ResNet 50 | 2016 [49] | MSR | 75.3 |
ResNet 101 | 2016 [49] | MSR | 76.4 |
DenseNet201 | 2016 [50] | – | 77.0 |
Inception V2 | 2016 [51] | Google | 72.2 |
Inception V3 | 2016 [51] | Google | 76.9 |
Inception V4 | 2017 [52] | Google | 80.2 |
InceptionResNet V2 | 2017 [52] | Google | 79.79 |
Pre-trained CNN models
Inception-V3
Inception-ResNet-V2
DenseNet-201
Dataset
Dataset | Total images | Training/validation set | Testing set |
---|---|---|---|
\({PH}^{2}\) | 200 | 160 | 40 |
ISIC MSK | 225 | 180 | 45 |
ISIC UDA | 557 | 446 | 111 |
ISBI 2017 | 2750 | 2200 | 550 |
Proposed framework
Preprocessing
Lesion/image segmentation
Deep features extraction
Feature layers
Pre-trained model | FC-layer | Feature-vector notation |
---|---|---|
Densenet201 | fc1000 | FV0 |
Inception-Resnet V2 | Predictions | FV1 |
Inception V3 | Predictions | FV2 |
Fusion mechanism
Entropy-controlled NCA
Feature selection
Dimensionality reduction
Results and discussion
Classifier | Base parameters |
---|---|
Fine tree | Maximum splits: 100 split criterion: Gini’s Diversity Index |
Medium tree | Maximum splits: 20 split criterion: Gini’s Diversity Index |
Coarse tree | Maximum splits: 4 split criterion: Gini’s Diversity Index |
Linear SVM | Kernel function: linear multi-class method: one-vs-one |
Q-SVM | Kernel function: quadratic multi-class method: one-vs-one |
Cubic SVM | Kernel function: cubic multi-class method: one-vs-one |
Fine KNN | Number of neighbors: 1 distance metric: Euclidean distance weight: equal |
Medium KNN | Number of neighbors: 10 distance metric: Euclidean distance weight: equal |
W-KNN | Number of neighbors: 10 distance metric: Euclidean distance weight: squared inverse |
Ensemble-BT | Ensemble method: AdaBoost learner type: decision tree maximum splits: 20 number of learners: 30 |
Ensemble subset KNN | Ensemble method: subspace learner type: nearest neighbor number of learners: 30 |
Ensemble RUSB | Ensemble method: RUSBoost learner type: decision tree number of learners: 30 maximum splits: 20 |
Evaluation of the single layer features
Evaluation of the proposed technique
Vector fusion | Input dimension | Output dimension | Percentage reduction (%) |
---|---|---|---|
\({PH}^{2}\) | |||
FV0–FV1 | 160 × 2000 | 160 × 50 | 97.50 |
FV0–FV2 | 160 × 2000 | 160 × 53 | 97.35 |
FV1–FV2 | 160 × 2000 | 160 × 55 | 97.25 |
FV0–FV1–FV2 | 160 × 3000 | 160 × 45 | 98.50* |
ISIC-MSK | |||
FV0–FV1 | 180 × 2000 | 180 × 99 | 95.05 |
FV0–FV2 | 180 × 2000 | 180 × 125 | 93.75 |
FV1–FV2 | 180 × 2000 | 180 × 167 | 91.65 |
FV0–FV1–FV2 | 180 × 3000 | 180 × 95 | 96.83 |
ISIC-UDA | |||
FV0–FV1 | 446 × 2000 | 446 × 227 | 88.65 |
FV0–FV2 | 446 × 2000 | 446 × 197 | 90.15 |
FV1–FV2 | 446 × 2000 | 446 × 161 | 91.95 |
FV0–FV1–FV2 | 446 × 3000 | 446 × 104 | 96.53 |
ISBI-2017 | |||
FV0–FV1 | 2200 × 2000 | 2200 × 107 | 95.13 |
FV0–FV2 | 2200 × 2000 | 2200 × 65 | 97.05 |
FV1–FV2 | 2200 × 2000 | 2200 × 53 | 97.59 |
FV0–FV1–FV2 | 2200 × 3000 | 2200 × 49 | 98.37 |
- Case 1: on \(PH^{2}\) dataset, the best classification accuracy achieved is 83.2% using Fine KNN (F-KNN), 82.2% using SVM and 82.4% using ES-KNN classifier, when FV0–FV1–FV2 are fused. Similarly, on ISIC-MSK dataset, by using the same fusion, F-KNN outperforms SVM and ES-KNN by achieving 76.4%. In case of ISIC-UDA, F-KNN yields 76.5% classification accuracy, which is greater than SVM (73.5%) and ES-KNN (76.0%). On ISBI-2017 dataset ES-KNN gives 76.1% accuracy, which is greater than both SVM and F-KNN. It has been observed, and hence concluded, that irrespective of the given dataset, the best classification results are obtained with the fusion of FV0–FV1–FV2, thereby validating the strength of the feature fusion approach.
- Case 2: using entropy-controlled feature fusion approach, on \(PH^{2}\), ISIC-MSK, and ISIC-UDA datasets, F-KNN yields the best accuracy of 98.8%, 99.2%, and 97.1% respectively, courtesy the feature fusion approach. In case of ISB1-2017 dataset, however, ES-KNN gives maximum accuracy of 95.9%. Note that the number of image samples in ISBI-2017 is larger as compared to other datasets; it may be concluded that ES-KNN gives classification results better as compared to other classifiers for datasets having greater number of samples.
Vector Fusion | OA (%) | |||||
---|---|---|---|---|---|---|
Feature Fusion Approach | Proposed (ECNCA) | |||||
F-KNN | SVM | ES-KNN | F-KNN | SVM | ES-KNN | |
\(\hbox {PH}^{2}\) | ||||||
FV0-FV1 | 82.8 | 80.0 | 80.1 | 96.9 | 93.7 | 95.1 |
FV0-FV2 | 82.1 | 81.7 | 81.7 | 95.1 | 94.8 | 93.2 |
FV1-FV2 | 82.9 | 82.0 | 82.1 | 97.4 | 95.0 | 97.1 |
FV0-FV1-FV2 | 83.2 | 82.2 | 82.4 | \(98.8*\) | 95.1 | 98.1 |
ISIC-MSK | ||||||
FV0-FV1 | 74.2 | 71.7 | 74.6 | 93.7 | 87.2 | 88.8 |
FV0-FV2 | 73.9 | 71.0 | 73.0 | 89.1 | 89.0 | 87.4 |
FV1-FV2 | 73.1 | 72.5 | 75.1 | 86.5 | 91.0 | 89.7 |
FV0-FV1-FV2 | 76.4 | 74.8 | 74.9 | \(99.2*\) | 95.1 | 96.9 |
ISIC-UDA | ||||||
FV0-FV1 | 71.9 | 70.0 | 75.9 | 88.8 | 80.1 | 84.7 |
FV0-FV2 | 73.3 | 71.2 | 74.1 | 90.7 | 84.5 | 88.0 |
FV1-FV2 | 74.1 | 75.9 | 75.8 | 92.8 | 82.7 | 94.2 |
FV0-FV1-FV2 | 76.5 | 73.5 | 76.0 | \(97.1*\) | 93.3 | 95.7 |
ISBI-2017 | ||||||
FV0-FV1 | 73.2 | 70.9 | 71.1 | 88.5 | 88.0 | 88.8 |
FV0-FV2 | 74.7 | 70.7 | 72.8 | 89.7 | 87.3 | 89.3 |
FV1-FV2 | 72.1 | 70.5 | 73.3 | 90.0 | 88.9 | 90.7 |
FV0-FV1-FV2 | 75.3 | 75.1 | 76.1 | 94.1 | 93.4 | \(95.9*\) |
Dataset | Average classification time (s) | Average accuracy | ||
---|---|---|---|---|
Entropy-controlled | Simple fusion | Entropy-controlled | Simple fusion | |
PH\(^2\) | 0.60 | 36 | 98.80 | 83.20 |
ISIC-MSK | 0.73 | 43 | 99.20 | 74.40 |
ISIC-UDA | 1.62 | 96 | 97.10 | 76.50 |
ISBI-2017 | 7.59 | 455 | 95.90 | 76.10 |
Classifier | Dataset | Performance Measures | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
I | II | III | IV | OA (%) | Recall | Precision | FNR | FPR | AUC | Time (sec) | |
Fine Tree |
\(\checkmark \)
| 87.5 | 83.0 | 80.5 | 12.5 | 0.17 | 0.84 | 0.76 | |||
\(\checkmark \)
| 79.1 | 78.5 | 79.0 | 20.9 | 0.22 | 0.73 | 0.93 | ||||
\(\checkmark \)
| 82.6 | 72.9 | 72.9 | 17.4 | 0.34 | 0.69 | 1.87 | ||||
\(\checkmark \)
| 89.3 | 77.9 | 78.9 | 10.2 | 0.20 | 0.73 | 5.04 | ||||
Medium Tree |
\(\checkmark \)
| 87.5 | 83.0 | 80.5 | 12.5 | 0.17 | 0.84 | 0.58 | |||
\(\checkmark \)
| 79.1 | 78.5 | 79.0 | 20.9 | 0.22 | 0.73 | 0.76 | ||||
\(\checkmark \)
| 84.2 | 74.9 | 74.4 | 15.8 | 0.32 | 0.72 | 1.70 | ||||
\(\checkmark \)
| 87.8 | 69.4 | 75.4 | 11.7 | 0.29 | 0.73 | 2.16 | ||||
Coarse Tree |
\(\checkmark \)
| 86.9 | 78.0 | 80.0 | 13.1 | 0.22 | 0.77 | 0.59 | |||
\(\checkmark \)
| 80.4 | 79.0 | 80.5 | 19.6 | 0.22 | 0.74 | 0.73 | ||||
\(\checkmark \)
| 87.1 | 70.9 | 71.4 | 12.9 | 0.36 | 0.67 | 1.60 | ||||
\(\checkmark \)
| 87.5 | 68.4 | 74.4 | 12 | 0.30 | 0.67 | 2.06 | ||||
Linear SVM |
\(\checkmark \)
| 93.1 | 87.5 | 90.5 | 6.9 | 0.12 | 0.97 | 0.60 | |||
\(\checkmark \)
| 88.7 | 88.0 | 88.5 | 11.3 | 0.13 | 0.91 | 0.78 | ||||
\(\checkmark \)
| 95.2 | 80.9 | 90.9 | 4.8 | 0.26 | 0.91 | 1.78 | ||||
\(\checkmark \)
| 91.3 | 72.9 | 84.9 | 8.2 | 0.25 | 0.84 | 4.3 | ||||
Quadratic SVM |
\(\checkmark \)
| 95.6 | 93.0 | 93.5 | 4.4 | 0.07 | 0.99 | 0.61 | |||
\(\checkmark \)
| 90.9 | 90.5 | 91.0 | 9.1 | 0.10 | 0.93 | 0.78 | ||||
\(\checkmark \)
| 96.2 | 87.9 | 91.9 | 3.8 | 0.19 | 0.94 | 1.71 | ||||
\(\checkmark \)
| 94.0 | 82.4 | 88.4 | 5.5 | 0.20 | 0.88 | 4.5 | ||||
Cubic SVM |
\(\checkmark \)
| 96.9 | 93.5 | 97.0 | 3.1 | 0.07 | 1.00 | 0.61 | |||
\(\checkmark \)
| 91.8 | 91.5 | 91.5 | 8.2 | 0.09 | 0.96 | 0.78 | ||||
\(\checkmark \)
| 96.6 | 89.4 | 92.9 | 3.4 | 0.18 | 0.95 | 1.69 | ||||
\(\checkmark \)
| 95.6 | 87.4 | 89.4 | 3.9 | 0.21 | 0.90 | 5.42 | ||||
F-KNN |
\(\checkmark \)
|
\(98.8*\)
| 97.0 | 99.0 | 1.2 | 0.03 | 0.97 | 0.60 | |||
\(\checkmark \)
|
\(99.2*\)
| 99.0 | 99.0 | 0.8 | 0.02 | 0.94 | 0.73 | ||||
\(\checkmark \)
|
\(97.1*\)
| 93.9 | 93.4 | 2.9 | 0.13 | 0.88 | 1.62 | ||||
\(\checkmark \)
| 95.8 | 92.9 | 92.4 | 4.6 | 0.19 | 0.86 | 2.53 | ||||
Medium KNN |
\(\checkmark \)
| 92.5 | 81.5 | 95.5 | 7.5 | 0.19 | 1.00 | 0.63 | |||
\(\checkmark \)
| 91.8 | 91.0 | 91.5 | 8.2 | 0.10 | 0.96 | 0.72 | ||||
\(\checkmark \)
| 95.5 | 78.9 | 91.9 | 4.5 | 0.28 | 0.90 | 1.50 | ||||
\(\checkmark \)
| 89.3 | 62.4 | 91.4 | 10.2 | 0.35 | 0.86 | 2.15 | ||||
Weighted KNN |
\(\checkmark \)
| 93.1 | 83.0 | 96.0 | 6.9 | 0.17 | 1.00 | 0.62 | |||
\(\checkmark \)
| 94.4 | 93.5 | 95.0 | 5.6 | 0.07 | 0.98 | 0.72 | ||||
\(\checkmark \)
| 95.2 | 80.9 | 90.9 | 4.8 | 0.26 | 0.92 | 1.60 | ||||
\(\checkmark \)
| 94.1 | 75.9 | 87.9 | 5.4 | 0.22 | 0.91 | 2.12 | ||||
Ensemble BT |
\(\checkmark \)
| 80.0 | 50.0 | 40.0 | 20 | 0.20 | 0.90 | 0.78 | |||
\(\checkmark \)
| 82.6 | 82.0 | 82.5 | 17.4 | 0.19 | 0.83 | 3.47 | ||||
\(\checkmark \)
| 93.2 | 81.4 | 86.4 | 6.8 | 0.26 | 0.85 | 7.87 | ||||
\(\checkmark \)
| 92.3 | 73.4 | 88.9 | 7.2 | 0.25 | 0.87 | 13.48 | ||||
Ensemble S-KNN |
\(\checkmark \)
| 98.1 | 95.5 | 99.0 | 1.9 | 0.04 | 1.00 | 4.06 | |||
\(\checkmark \)
| 96.2 | 96.0 | 96.0 | 3.8 | 0.05 | 0.99 | 3.49 | ||||
\(\checkmark \)
| 96.8 | 89.9 | 91.4 | 3.2 | 0.17 | 0.92 | 5.36 | ||||
\(\checkmark \)
|
\(95.9*\)
| 93.4 | 95.4 | 3.6 | 0.17 | 0.92 | 7.59 | ||||
Ensemble RUSB |
\(\checkmark \)
| 88.8 | 86.0 | 81.5 | 11.2 | 0.14 | 0.93 | 4.74 | |||
\(\checkmark \)
| 85.7 | 85.0 | 85.5 | 14.3 | 0.16 | 0.88 | 5.24 | ||||
\(\checkmark \)
| 85.5 | 87.9 | 83.4 | 14.5 | 0.21 | 0.88 | 7.09 | ||||
\(\checkmark \)
| 83.3 | 82.4 | 74.9 | 16.2 | 0.20 | 0.85 | 9.45 |
Comparison with state of the art techniques
Ref | Year | Dataset | Method | OA (%) |
---|---|---|---|---|
[65] | 2016 | \(PH^{2}\) | ABCD rule | 90.00 |
[66] | 2016 | \(PH^{2}\) | wavelet transform with morphological operations | 93.87 |
[15] | 2017 | \(PH^{2}\) | multistage fully convolutional network | 94.24 |
[67] | 2017 | \(PH^{2}\) | color and texture features | 96.00 |
[68] | 2018 | ISBI-2017 | regularised discriminant learning | 83.20 |
[13] | 2018 | ISBI-2017 | fully convolutional residual networks & lesion index calculation unit | 85.70 |
[69] | 2018 | ISBI-2017 | Ensemble Of Deep Neural Networks | 84.8% |
[18] | 2018 | ISIC-MSK | probabilistic distribution and best features selection | 97.20 |
Proposed | 2019 | ISBI-2017 | ECNCA | 95.90 |
Proposed | 2019 | ISIC-UDA | ECNCA | 97.10 |
Proposed | 2019 | ISIC-MSK | ECNCA | 99.20 |
Proposed | 2019 | \(PH^{2}\) | ECNCA | 98.80 |