Introduction
Methods
Object detection algorithm
Method overview
Implementation details
Object detection metrics
Localization criterion | Classification metric | |
---|---|---|
EndoVis 2015 [16] | Point inside mask | PPV |
Sensitivity | ||
Overall/Average F1-score | ||
GIANA 2017 [6] | Point inside mask | PPV |
GIANA 2018 [6] | Sensitivity | |
F1/F2-score | ||
Custom metrics | ||
GIANA 2021 [5] | Box IoU (only 2021) | AP@0.5:0.95 (only 2021) |
EndoCV 2021 [3] | Box IoU | AP@0.5, AP@0.75, AP@0.5:0.95 |
EndoCV 2022 [4] | AP across 3 scales of polyp size | |
Mean of 4 APs |
Experiments and results
Effect of test set
Sensitivity to center
Sensitivity to polyp size
Metric | AP@0.5 | AP@0.5:0.95 | n | \(\phi (\%) \) | ||||
---|---|---|---|---|---|---|---|---|
Polyp size | Small | Medium | Large | Small | Medium | Large | ||
Center 1 | 0.24 | 0.48 | 0.73 | 0.14 | 0.33 | 0.55 | 256 | 98 |
Center 2 | 0.00 | 0.46 | 0.81 | 0.00 | 0.31 | 0.67 | 276 | 89 |
Center 3 | 0.16 | 0.64 | 0.91 | 0.10 | 0.47 | 0.72 | 457 | 99 |
Center 4 | 0.14 | 0.33 | 0.59 | 0.06 | 0.27 | 0.43 | 227 | 64 |
Center 5 | 0.19 | 0.39 | 0.59 | 0.12 | 0.26 | 0.42 | 208 | 99 |
Center 6 | 0.00 | 0.52 | 0.89 | 0.00 | 0.39 | 0.67 | 88 | 94 |
All centers (SD) | 0.14 (0.1) | 0.47 (0.11) | 0.75 (0.14) | 0.07 (0.06) | 0.34 (0.08) | 0.58 (0.13) | 1512 | 91 |
Effect of metric configuration
Sensitivity of the AP to the specific choice of overlap-based localization criterion
Sensitivity of the AP to the IoU range
IoU versus point in mask
Localization criterion | Box IoU = 0.5 | Point inside box | Point inside mask | Point inside hull |
---|---|---|---|---|
Sensitivity | 0.68 | 0.74 | 0.74 | 0.74 |
PPV | 0.78 | 0.86 | 0.85 | 0.86 |
F1-Score | 0.73 | 0.8 | 0.79 | 0.8 |
F2-Score | 0.70 | 0.74 | 0.74 | 0.74 |
AP | 0.65 | 0.73 | 0.73 | 0.73 |
Alignment with clinical interest
Generalizability of results
Discussion
Localization criterion | Box IoU = 0.5 | Point inside box | Point inside mask | Point inside hull |
---|---|---|---|---|
EfficientDet ensemble (Complementary method #1) | ||||
Sensitivity | 0.49 | 0.61 | 0.59 | 0.59 |
PPV | 0.36 | 0.46 | 0.44 | 0.44 |
F1-Score | 0.41 | 0.51 | 0.49 | 0.49 |
F2-Score | 0.45 | 0.56 | 0.54 | 0.54 |
AP | 0.37 | 0.49 | 0.47 | 0.47 |
YOLOv7 ensemble (Complementary method #2) | ||||
Sensitivity | 0.64 | 0.76 | 0.75 | 0.76 |
PPV | 0.71 | 0.78 | 0.77 | 0.77 |
F1-Score | 0.67 | 0.77 | 0.76 | 0.76 |
F2-Score | 0.65 | 0.77 | 0.76 | 0.76 |
AP | 0.59 | 0.74 | 0.73 | 0.73 |
Metric |
AP@0.5
|
AP@0.5:0.95
| ||||
---|---|---|---|---|---|---|
Polyp size | Small | Medium | Large | Small | Medium | Large |
EfficientDet ensemble (Complementary method #1)
| ||||||
Center 1 | 0.12 | 0.2 | 0.5 | 0.07 | 0.08 | 0.22 |
Center 2 | 0.00 | 0.39 | 0.65 | 0.00 | 0.25 | 0.38 |
Center 3 | 0.02 | 0.24 | 0.59 | 0.01 | 0.12 | 0.29 |
Center 4 | 0.03 | 0.0 | 0.24 | 0.02 | 0.0 | 0.13 |
Center 5 | 0.08 | 0.08 | 0.17 | 0.03 | 0.04 | 0.08 |
Center 6 | 0.00 | 0.03 | 0.56 | 0.00 | 0.23 | 0.32 |
All centers (SD) | 0.04 (0.05) | 0.2 (0.14) | 0.45 (0.2) | 00.2 (0.03) | 0.12 (0.1) | 0.24 (0.12) |
YOLOv7 ensemble (Complementary method #2)
| ||||||
Center 1 | 0.12 | 0.41 | 0.59 | 0.07 | 0.29 | 0.45 |
Center 2 | 0.00 | 0.53 | 0.81 | 0.00 | 0.33 | 0.63 |
Center 3 | 0.12 | 0.57 | 0.83 | 0.06 | 0.36 | 0.64 |
Center 4 | 0.07 | 0.49 | 0.58 | 0.01 | 0.26 | 0.44 |
Center 5 | 0.22 | 0.41 | 0.58 | 0.13 | 0.27 | 0.39 |
Center 6 | 0.00 | 0.46 | 0.69 | 0.00 | 0.3 | 0.55 |
All centers (SD) | 0.09 (0.08) | 0.48 (0.06) | 0.68 (0.12) | 0.05 (0.05) | 0.3 (0.04) | 0.52 (0.11) |