nach oben

Erschienen in:

2021 | OriginalPaper | Buchkapitel

Fighting Class Imbalance with Contrastive Learning

verfasst von : Yassine Marrakchi, Osama Makansi, Thomas Brox

Erschienen in: Medical Image Computing and Computer Assisted Intervention – MICCAI 2021

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Medical image datasets are hard to collect, expensive to label, and often highly imbalanced. The last issue is underestimated, as typical average metrics hardly reveal that the often very important minority classes have a very low accuracy. In this paper, we address this problem by a feature embedding that balances the classes using contrastive learning as an alternative to the common cross-entropy loss. The approach is largely orthogonal to existing sampling methods and can be easily combined with those. We show on the challenging ISIC2018 and APTOS2019 datasets that the approach improves especially the accuracy of minority classes without negatively affecting the majority ones.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel The Power of Proxy Data and Proxy Networks for Hyper-parameter Optimization in Medical Image Segmentation

Nächstes Kapitel Interpretable Gender Classification from Retinal Fundus Images Using BagNets

Aptos 2019 blindness detection (2019). https://www.kaggle.com/c/aptos2019-blindness-detection/data

Aleksey Nozdryn-Plotnicki, J.Y., Yolland, W.: Ensembling convolutional neural networks for skin cancer classification. ArXiv (2018)

Cao, K., Wei, C., Gaidon, A., Aréchiga, N., Ma, T.: Learning imbalanced datasets with label-distribution-aware margin loss. In: NeurIPS (2019)

Chan, H.P., Samala, R.K., Hadjiiski, L.M., Zhou, C.: Deep learning in medical image analysis (2020)

Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. JAIR 16, 321–357 (2002)CrossRef

Chen, T., Kornblith, S., Norouzi, M., Hinton, G.E.: A simple framework for contrastive learning of visual representations. In: ICML (2020)

Codella, N.C.F., et al.: Skin lesion analysis toward melanoma detection 2018: a challenge hosted by the international skin imaging collaboration (ISIC). CoRR abs/1902.03368 (2019)

Cui, Y., Jia, M., Lin, T., Song, Y., Belongie, S.J.: Class-balanced loss based on effective number of samples. In: CVPR (2019)

Doi, K.: Computer-aided diagnosis in medical imaging: historical review, current status and future potential. Comput. Med. Imaging Graph. 31, 198–211 (2007)CrossRef

10.

Dong, Q., Gong, S., Zhu, X.: Imbalanced deep learning by minority class incremental rectification. IEEE TPAMI 41, 1367–1381 (2019)CrossRef

11.

Dosovitskiy, A., Fischer, P., Springenberg, J.T., Riedmiller, M.A., Brox, T.: Discriminative unsupervised feature learning with exemplar convolutional neural networks. IEEE TPAMI, 1734–1747 (2016)

12.

Drummond, C., Holte, R.: C4.5, class imbalance, and cost sensitivity: why under-sampling beats oversampling. In: ICML Workshop (2003)

13.

Fotedar, G., Tajbakhsh, N., Ananth, S., Ding, X.: Extreme consistency: overcoming annotation scarcity and domain shifts. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 699–709. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_68CrossRef

14.

Gong, L., Ma, K., Zheng, Y.: Distractor-aware neuron intrinsic learning for generic 2D medical image classifications. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 591–601. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_57CrossRef

15.

Guanjin Wang, K.W.W., Lu, J.: AUC-based extreme learning machines for supervised and semi-supervised imbalanced classification. IEEE Trans. Syst. Man Cybern.: Syst., 1–12 (2020)

16.

Gutmann, M., Hyvärinen, A.: Noise-contrastive estimation: a new estimation principle for unnormalized statistical models. In: AISTATS (2010)

17.

Gyawali, P.K., Ghimire, S., Bajracharya, P., Li, Z., Wang, L.: Semi-supervised medical image classification with global latent mixing. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 604–613. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_59CrossRef

18.

Han, H., Wang, W.-Y., Mao, B.-H.: Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning. In: Huang, D.-S., Zhang, X.-P., Huang, G.-B. (eds.) ICIC 2005. LNCS, vol. 3644, pp. 878–887. Springer, Heidelberg (2005). https://doi.org/10.1007/11538059_91CrossRef

19.

Hayat, M., Khan, S., Zamir, S.W., Shen, J., Shao, L.: Gaussian affinity for max-margin class imbalanced learning. In: ICCV (2019)

20.

He, H., Garcia, E.A.: Learning from imbalanced data. IEEE TKDE 21, 1263–1284 (2009)

21.

He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)

22.

Huang, C., Li, Y., Loy, C.C., Tang, X.: Learning deep representation for imbalanced classification. In: CVPR (2016)

23.

Kang, B., et al.: Decoupling representation and classifier for long-tailed recognition. In: ICLR (2020)

24.

Khan, S.H., Hayat, M., Zamir, S.W., Shen, J., Shao, L.: Striking the right balance with uncertainty. In: CVPR (2019)

25.

Khosla, P., et al.: Supervised contrastive learning. In: NeurIPS (2020)

26.

Kim, J., Jeong, J., Shin, J.: M2m: imbalanced classification via major-to-minor translation. In: CVPR (2020)

27.

Li, X., Hu, X., Yu, L., Zhu, L., Fu, C.W., Heng, P.: CANet: cross-disease attention network for joint diabetic retinopathy and diabetic macular edema grading. IEEE Trans. Med. Imaging, 1483–1493 (2020)

28.

Li, X., Yu, L., Jin, Y., Fu, C.-W., Xing, L., Heng, P.-A.: Difficulty-aware meta-learning for rare disease diagnosis. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 357–366. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_35CrossRef

29.

Li, Z., Zhong, C., Wang, R., Zheng, W.-S.: Continual learning of new diseases with dual distillation and ensemble strategy. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 169–178. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_17CrossRef

30.

Liao, H., Luo, J.: A deep multi-task learning approach to skin lesion classification. In: AAAI workshop (2017)

31.

Lin, T., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: ICCV (2017)

32.

Loshchilov, I., Hutter, F.: SGDR: stochastic gradient descent with warm restarts. In: ICLR (2017)

33.

Mullick, S.S., Datta, S., Das, S.: Generative adversarial minority oversampling. In: ICCV (2019)

34.

van den Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. CoRR abs/1807.03748 (2018)

35.

Peilin Zhao, Steven C. H. Hoi, R.J., Yang, T.: Online AUC maximization. In: ICML (2011)

36.

Peng, J., Bu, X., Sun, M., Zhang, Z., Tan, T., Yan, J.: Large-scale object detection in the wild from imbalanced multi-labels. In: CVPR (2020)

37.

Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2015)MathSciNetCrossRef

38.

Shen, L., Lin, Z., Huang, Q.: Relay backpropagation for effective learning of deep convolutional neural networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9911, pp. 467–482. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46478-7_29CrossRef

39.

Shrivastava, A., Gupta, A., Girshick, R.B.: Training region-based object detectors with online hard example mining. In: CVPR (2016)

40.

Shrivastava, A., Gupta, A., Girshick, R: Training region-based object detectors with online hard example mining. In: CVPR (2016)

41.

Skorczyk-Werner, A., et al.: Fundus albipunctatus: review of the literature and report of a novel RDH5 gene mutation affecting the invariant tyrosine (p. Tyr175Phe). J. Appl. Genet. 56, 317–327 (2015)CrossRef

42.

Wei, D., Cao, S., Ma, K., Zheng, Y.: Learning and exploiting interclass visual correlations for medical image classification. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 106–115. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_11CrossRef

43.

Zhang, X., Fang, Z., Wen, Y., Li, Z., Qiao, Y.: Range loss for deep face recognition with long-tailed training data. In: ICCV (2017)

44.

Zhuang, J.X., et al.: Skin lesion analysis towards melanoma detection using deep neural network ensemble (2018)

45.

Zhuang, J., Cai, J., Wang, R., Zhang, J., Zheng, W.-S.: Deep kNN for medical image classification. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 127–136. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_13CrossRef

Titel: Fighting Class Imbalance with Contrastive Learning
verfasst von: Yassine Marrakchi
Osama Makansi
Thomas Brox
Verlag: Springer International Publishing
Buch: Medical Image Computing and Computer Assisted Intervention – MICCAI 2021
Print ISBN: 978-3-030-87198-7

Electronic ISBN: 978-3-030-87199-4

Copyright-Jahr: 2021
DOI: https://doi.org/10.1007/978-3-030-87199-4_44

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"