Skip to main content
Erschienen in: Multimedia Systems 5/2023

14.07.2023 | Regular Paper

HCPSNet: heterogeneous cross-pseudo-supervision network with confidence evaluation for semi-supervised medical image segmentation

verfasst von: Xianhua Duan, Chaoqiang Jin, Xin Shu

Erschienen in: Multimedia Systems | Ausgabe 5/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Medical image segmentation technology can effectively help doctors to diagnose, but there are too little annotated data, which limits the development of fully supervised medical image segmentation methods. This dilemma leads to urgent research on semi-supervised medical image segmentation methods. To cope with this dilemma, we propose a semi-supervised dual flow network, which is called the Heterogeneous Cross-pseudo-supervision Network (HCPSNet). In the HCPSNet, Unet and Swin-Unet are combined for cross-learning, and a shifted patch tokenization (SPT) module is embedded into Swin-Unet to increase the spatial information contained in the feature maps. Besides, a confidence evaluation (CE) module is present to improve the performance of the model. The experimental results on three medical clinical datasets, LA2018, BraTs2019, and ACDC, show that our method can achieve good segmentation results with limited labeled samples. The mean dice of our proposed network on ACDC with seven cases’ samples is 86.17%, about 3% higher than other models.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Hollon, T.C., Pandian, B., Adapa, A.R., et al.: Near real-time intraoperative brain tumor diagnosis using stimulated Raman histology and deep neural networks. Nat. Med. 26(1), 52–58 (2020)CrossRef Hollon, T.C., Pandian, B., Adapa, A.R., et al.: Near real-time intraoperative brain tumor diagnosis using stimulated Raman histology and deep neural networks. Nat. Med. 26(1), 52–58 (2020)CrossRef
2.
Zurück zum Zitat Litjens, G., Kooi, T., Bejnordi, B.E., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)CrossRef Litjens, G., Kooi, T., Bejnordi, B.E., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)CrossRef
3.
Zurück zum Zitat Shen, D., Wu, G., Suk, H.I.: Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19, 221 (2017)CrossRef Shen, D., Wu, G., Suk, H.I.: Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19, 221 (2017)CrossRef
4.
Zurück zum Zitat Hesamian, M.H., Jia, W., He, X., et al.: Deep learning techniques for medical image segmentation: achievements and challenges. J. Digit. Imaging 32(4), 582–596 (2019)CrossRef Hesamian, M.H., Jia, W., He, X., et al.: Deep learning techniques for medical image segmentation: achievements and challenges. J. Digit. Imaging 32(4), 582–596 (2019)CrossRef
5.
Zurück zum Zitat Li, X., Chen, H., Qi, X., et al.: H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans. Med. Imaging 37(12), 2663–2674 (2018)CrossRef Li, X., Chen, H., Qi, X., et al.: H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans. Med. Imaging 37(12), 2663–2674 (2018)CrossRef
6.
Zurück zum Zitat Yang, Z., Xie, L., Zhou, W., et al.: VoxSeP: semi-positive voxels assist self-supervised 3D medical segmentation. Multimed. Syst. 4, 1–16 (2022) Yang, Z., Xie, L., Zhou, W., et al.: VoxSeP: semi-positive voxels assist self-supervised 3D medical segmentation. Multimed. Syst. 4, 1–16 (2022)
7.
Zurück zum Zitat Liu, Z., Han, K., Wang, Z., et al.: Automatic liver segmentation from abdominal CT volumes using improved convolution neural networks. Multimed. Syst. 27(1), 111–124 (2021)CrossRef Liu, Z., Han, K., Wang, Z., et al.: Automatic liver segmentation from abdominal CT volumes using improved convolution neural networks. Multimed. Syst. 27(1), 111–124 (2021)CrossRef
8.
Zurück zum Zitat Bai, W., Suzuki, H., Huang, J., et al.: A population-based phenome-wide association study of cardiac and aortic structure and function. Nat. Med. 26(10), 1654–1662 (2020)CrossRef Bai, W., Suzuki, H., Huang, J., et al.: A population-based phenome-wide association study of cardiac and aortic structure and function. Nat. Med. 26(10), 1654–1662 (2020)CrossRef
9.
Zurück zum Zitat Mei, X., Lee, H.C., Diao, K., et al.: Artificial intelligence-enabled rapid diagnosis of patients with COVID-19. Nat. Med. 26(8), 1224–1228 (2020)CrossRef Mei, X., Lee, H.C., Diao, K., et al.: Artificial intelligence-enabled rapid diagnosis of patients with COVID-19. Nat. Med. 26(8), 1224–1228 (2020)CrossRef
10.
Zurück zum Zitat Olimov, B., Sanjar, K., Din, S., et al.: FU-Net: fast biomedical image segmentation model based on bottleneck convolution layers. Multimed. Syst. 27(4), 637–650 (2021)CrossRef Olimov, B., Sanjar, K., Din, S., et al.: FU-Net: fast biomedical image segmentation model based on bottleneck convolution layers. Multimed. Syst. 27(4), 637–650 (2021)CrossRef
11.
Zurück zum Zitat Peixoto, S.A., Medeiros, A.G., Hassan, M.M., et al.: Floor of log: a novel intelligent algorithm for 3D lung segmentation in computer tomography images. Multimed. Syst. 7, 1–13 (2020) Peixoto, S.A., Medeiros, A.G., Hassan, M.M., et al.: Floor of log: a novel intelligent algorithm for 3D lung segmentation in computer tomography images. Multimed. Syst. 7, 1–13 (2020)
12.
Zurück zum Zitat Olya, M.H., Badri, H., Teimoori, S., et al.: An integrated deep learning and stochastic optimization approach for resource management in team-based healthcare systems. Exp. Syst. Appl. 187, 115924 (2022)CrossRef Olya, M.H., Badri, H., Teimoori, S., et al.: An integrated deep learning and stochastic optimization approach for resource management in team-based healthcare systems. Exp. Syst. Appl. 187, 115924 (2022)CrossRef
13.
Zurück zum Zitat Li, Z., Zhao, S., Chen, Y., et al.: A deep-learning-based framework for severity assessment of COVID-19 with CT images. Exp. Syst. Appl. 185, 115616 (2021)CrossRef Li, Z., Zhao, S., Chen, Y., et al.: A deep-learning-based framework for severity assessment of COVID-19 with CT images. Exp. Syst. Appl. 185, 115616 (2021)CrossRef
14.
Zurück zum Zitat Zhang, H., Zhang, W., Shen, W., et al.: Automatic segmentation of the cardiac MR images based on nested fully convolutional dense network with dilated convolution. Biomed. Signal Process. Control 68, 102684 (2021)CrossRef Zhang, H., Zhang, W., Shen, W., et al.: Automatic segmentation of the cardiac MR images based on nested fully convolutional dense network with dilated convolution. Biomed. Signal Process. Control 68, 102684 (2021)CrossRef
15.
Zurück zum Zitat Wang, W., Wang, Y., Wu, Y., et al.: Quantification of full left ventricular metrics via deep regression learning with contour-guidance. IEEE Access 7, 47918–47928 (2019)CrossRef Wang, W., Wang, Y., Wu, Y., et al.: Quantification of full left ventricular metrics via deep regression learning with contour-guidance. IEEE Access 7, 47918–47928 (2019)CrossRef
16.
Zurück zum Zitat Shu, X., Chang, F., Zhang, X., et al.: ECAU-Net: efficient channel attention U-Net for fetal ultrasound cerebellum segmentation. Biomed. Signal Process. Control 75, 103528 (2022)CrossRef Shu, X., Chang, F., Zhang, X., et al.: ECAU-Net: efficient channel attention U-Net for fetal ultrasound cerebellum segmentation. Biomed. Signal Process. Control 75, 103528 (2022)CrossRef
17.
Zurück zum Zitat Shu, X., Gu, Y., Zhang, X., et al.: FCRB U-net: a novel fully connected residual block U-net for fetal cerebellum ultrasound image segmentation. Comput. Biol. Med. 3, 105693 (2022)CrossRef Shu, X., Gu, Y., Zhang, X., et al.: FCRB U-net: a novel fully connected residual block U-net for fetal cerebellum ultrasound image segmentation. Comput. Biol. Med. 3, 105693 (2022)CrossRef
18.
Zurück zum Zitat Kiran, I., Raza, B., Ijaz, A., et al.: DenseRes-Unet: segmentation of overlapped/clustered nuclei from multi organ histopathology images. Comput. Biol. Med. 143, 105267 (2022)CrossRef Kiran, I., Raza, B., Ijaz, A., et al.: DenseRes-Unet: segmentation of overlapped/clustered nuclei from multi organ histopathology images. Comput. Biol. Med. 143, 105267 (2022)CrossRef
19.
Zurück zum Zitat Pi, J., Qi, Y., Lou, M., et al.: FS-UNet: mass segmentation in mammograms using an encoder-decoder architecture with feature strengthening. Comput. Biol. Med. 137, 104800 (2021)CrossRef Pi, J., Qi, Y., Lou, M., et al.: FS-UNet: mass segmentation in mammograms using an encoder-decoder architecture with feature strengthening. Comput. Biol. Med. 137, 104800 (2021)CrossRef
20.
Zurück zum Zitat Wang, Y., Huang, L., Wu, M., et al.: Multi-input adaptive neural network for automatic detection of cervical vertebral landmarks on X-rays. Comput. Biol. Med. 146, 105576 (2022)CrossRef Wang, Y., Huang, L., Wu, M., et al.: Multi-input adaptive neural network for automatic detection of cervical vertebral landmarks on X-rays. Comput. Biol. Med. 146, 105576 (2022)CrossRef
21.
Zurück zum Zitat Xiong, Z., Xia, Q., Hu, Z., et al.: A global benchmark of algorithms for segmenting the left atrium from late gadolinium-enhanced cardiac magnetic resonance imaging. Med. Image Anal. 67, 101832 (2021)CrossRef Xiong, Z., Xia, Q., Hu, Z., et al.: A global benchmark of algorithms for segmenting the left atrium from late gadolinium-enhanced cardiac magnetic resonance imaging. Med. Image Anal. 67, 101832 (2021)CrossRef
22.
Zurück zum Zitat Tarvainen, A., Valpola, H.: Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. Adv. Neural Inf. Process. Syst. 30, 112 (2017) Tarvainen, A., Valpola, H.: Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. Adv. Neural Inf. Process. Syst. 30, 112 (2017)
23.
Zurück zum Zitat Yu, L., Wang, S., Li, X., et al.: Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 605–613. Springer, Cham (2019). Yu, L., Wang, S., Li, X., et al.: Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 605–613. Springer, Cham (2019).
24.
Zurück zum Zitat Li, S., Zhang, C., He, X.: Shape-aware semi-supervised 3D semantic segmentation for medical images. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 552–561. Springer, Cham (2020). Li, S., Zhang, C., He, X.: Shape-aware semi-supervised 3D semantic segmentation for medical images. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 552–561. Springer, Cham (2020).
25.
Zurück zum Zitat Luo, X., Chen, J., Song, T. et al.: Semi-supervised medical image segmentation through dual-task consistency. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35(10), pp. 8801–8809 (2021). Luo, X., Chen, J., Song, T. et al.: Semi-supervised medical image segmentation through dual-task consistency. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35(10), pp. 8801–8809 (2021).
26.
Zurück zum Zitat Qiao, S., Shen, W., Zhang, Z. et al.: Deep co-training for semi-supervised image recognition. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 135–152 (2018). Qiao, S., Shen, W., Zhang, Z. et al.: Deep co-training for semi-supervised image recognition. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 135–152 (2018).
27.
Zurück zum Zitat Zhang, Y., Yang, L., Chen, J. et al.: Deep adversarial networks for biomedical image segmentation utilizing unannotated images. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 408–416. Springer, Cham (2017). Zhang, Y., Yang, L., Chen, J. et al.: Deep adversarial networks for biomedical image segmentation utilizing unannotated images. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 408–416. Springer, Cham (2017).
28.
Zurück zum Zitat Vu, T.H., Jain, H., Bucher, M. et al.: Advent: adversarial entropy minimization for domain adaptation in semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2517–2526 (2019). Vu, T.H., Jain, H., Bucher, M. et al.: Advent: adversarial entropy minimization for domain adaptation in semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2517–2526 (2019).
29.
Zurück zum Zitat Peng, J., Wang, P., Desrosiers, C., et al.: Self-paced contrastive learning for semi-supervised medical image segmentation with meta-labels. Adv. Neural. Inf. Process. Syst. 34, 16686–16699 (2021) Peng, J., Wang, P., Desrosiers, C., et al.: Self-paced contrastive learning for semi-supervised medical image segmentation with meta-labels. Adv. Neural. Inf. Process. Syst. 34, 16686–16699 (2021)
30.
Zurück zum Zitat Ke, Z., Qiu, D., Li, K. et al.: Guided collaborative training for pixel-wise semi-supervised learning. In: European Conference on Computer Vision, pp. 429–445. Springer, Cham (2020). Ke, Z., Qiu, D., Li, K. et al.: Guided collaborative training for pixel-wise semi-supervised learning. In: European Conference on Computer Vision, pp. 429–445. Springer, Cham (2020).
31.
Zurück zum Zitat Chen, X., Yuan, Y., Zeng, G. et al.: Semi-supervised semantic segmentation with cross pseudo-supervision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2613–2622 (2021). Chen, X., Yuan, Y., Zeng, G. et al.: Semi-supervised semantic segmentation with cross pseudo-supervision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2613–2622 (2021).
32.
Zurück zum Zitat Peiris, H., Chen, Z., Egan, G. et al.: Duo-SegNet: adversarial dual-views for semi-supervised medical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 428–438. Springer, Cham (2021). Peiris, H., Chen, Z., Egan, G. et al.: Duo-SegNet: adversarial dual-views for semi-supervised medical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 428–438. Springer, Cham (2021).
33.
Zurück zum Zitat Li, C., Liu, H.: Generative adversarial semi-supervised network for medical image segmentation. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 303–306. (IEEE, 2021). Li, C., Liu, H.: Generative adversarial semi-supervised network for medical image segmentation. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 303–306. (IEEE, 2021).
34.
Zurück zum Zitat Bortsova, G., Dubost, F., Hogeweg, L. et al.: Semi-supervised medical image segmentation via learning consistency under transformations. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 810–818. Springer, Cham (2019). Bortsova, G., Dubost, F., Hogeweg, L. et al.: Semi-supervised medical image segmentation via learning consistency under transformations. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 810–818. Springer, Cham (2019).
35.
Zurück zum Zitat Hang, W., Feng, W., Liang, S. et al.: Local and global structure-aware entropy regularized mean teacher model for 3D left atrium segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 562–571. Springer, Cham (2020). Hang, W., Feng, W., Liang, S. et al.: Local and global structure-aware entropy regularized mean teacher model for 3D left atrium segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 562–571. Springer, Cham (2020).
36.
Zurück zum Zitat Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241. Springer, Cham (2015). Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241. Springer, Cham (2015).
37.
Zurück zum Zitat Dosovitskiy, A., Beyer, L., Kolesnikov, A. et al.: An image is worth 16 × 16 words: transformers for image recognition at scale. arXiv preprint: arXiv:2010.11929 (2020). Dosovitskiy, A., Beyer, L., Kolesnikov, A. et al.: An image is worth 16 × 16 words: transformers for image recognition at scale. arXiv preprint: arXiv:​2010.​11929 (2020).
38.
Zurück zum Zitat Touvron, H., Cord, M., Douze, M. et al.: Training data-efficient image transformers & distillation through attention. In: International Conference on Machine Learning. PMLR, pp. 10347–10357 (2021). Touvron, H., Cord, M., Douze, M. et al.: Training data-efficient image transformers & distillation through attention. In: International Conference on Machine Learning. PMLR, pp. 10347–10357 (2021).
39.
Zurück zum Zitat Liu, Z., Lin, Y., Cao, Y. et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022 (2021). Liu, Z., Lin, Y., Cao, Y. et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022 (2021).
40.
Zurück zum Zitat Chen, J., Lu, Y., Yu, Q. et al.: Transunet: transformers make strong encoders for medical image segmentation. arXiv preprint: arXiv:2102.04306 (2021). Chen, J., Lu, Y., Yu, Q. et al.: Transunet: transformers make strong encoders for medical image segmentation. arXiv preprint: arXiv:​2102.​04306 (2021).
41.
Zurück zum Zitat Cao, H., Wang, Y., Chen, J. et al.: Swin-unet: Unet-like pure transformer for medical image segmentation. arXiv preprint arXiv:2105.05537 (2021). Cao, H., Wang, Y., Chen, J. et al.: Swin-unet: Unet-like pure transformer for medical image segmentation. arXiv preprint arXiv:​2105.​05537 (2021).
42.
Zurück zum Zitat Luo, X., Hu, M., Song, T. et al.: Semi-supervised medical image segmentation via cross teaching between CNN and transformer. arXiv preprint: arXiv:2112.04894 (2021). Luo, X., Hu, M., Song, T. et al.: Semi-supervised medical image segmentation via cross teaching between CNN and transformer. arXiv preprint: arXiv:​2112.​04894 (2021).
44.
Zurück zum Zitat Touvron, H., Cord, M., El-Nouby, A. et al.: Three things everyone should know about vision transformers. arXiv preprint: arXiv:2203.09795 (2022). Touvron, H., Cord, M., El-Nouby, A. et al.: Three things everyone should know about vision transformers. arXiv preprint: arXiv:​2203.​09795 (2022).
45.
Zurück zum Zitat Li, X., Luo, G., Wang, K.: Multi-Step Cascaded Networks for Brain Tumor Segmentation. International MICCAI Brainlesion Workshop, pp. 163–173. Springer, Cham (2019) Li, X., Luo, G., Wang, K.: Multi-Step Cascaded Networks for Brain Tumor Segmentation. International MICCAI Brainlesion Workshop, pp. 163–173. Springer, Cham (2019)
46.
Zurück zum Zitat Luo, X., Liao, W., Chen, J. et al.: Efficient semi-supervised gross target volume of nasopharyngeal carcinoma segmentation via uncertainty rectified pyramid consistency. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 318–329. Springer, Cham (2021). Luo, X., Liao, W., Chen, J. et al.: Efficient semi-supervised gross target volume of nasopharyngeal carcinoma segmentation via uncertainty rectified pyramid consistency. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 318–329. Springer, Cham (2021).
Metadaten
Titel
HCPSNet: heterogeneous cross-pseudo-supervision network with confidence evaluation for semi-supervised medical image segmentation
verfasst von
Xianhua Duan
Chaoqiang Jin
Xin Shu
Publikationsdatum
14.07.2023
Verlag
Springer Berlin Heidelberg
Erschienen in
Multimedia Systems / Ausgabe 5/2023
Print ISSN: 0942-4962
Elektronische ISSN: 1432-1882
DOI
https://doi.org/10.1007/s00530-023-01135-5

Weitere Artikel der Ausgabe 5/2023

Multimedia Systems 5/2023 Zur Ausgabe