Skip to main content
Top
Published in:

07-09-2023

Prototype Consistency Learning for Medical Image Segmentation by Cross Pseudo Supervision

Authors: Lu Xie, Weigang Li, Yongqiang Wang, Yuntao Zhao

Published in: Cognitive Computation | Issue 1/2024

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Due to the acquisition of anatomical/pathological labels is expensive and time-consuming, semi-supervised semantic segmentation is commonly utilized in medical image analysis. Previous studies have overlooked the high similarity of the pixels in medical images, resulting in many models cannot effectively distinguish the pixels of different categories. A new semi-supervised semantic segmentation framework based on prototype learning is proposed in this paper. It contains a feature extractor and a superpixel-based graph convolutional network (GCN). Two consistency loss functions are proposed in our paper. The prototype cyclic consistency loss is utilized to incorporate explicit guidance of the labeled data; the cross pseudo supervised loss is applied to make full use of the context information of the unlabeled data. We evaluate the effectiveness of our proposed method on two classical public medical image datasets (MC and JSRT). On MC dataset, the predicted IoU of our method is 94.92 ±0.5% with only 25% annotated data; on JSRT dataset, the MIoU of our method reaches 89.51 ±0.37% (with 25% annotated data) and 90.98 ±0.4% (with 50% annotated data). Our proposed method outperforms most existing semi-supervised semantic segmentation methods, even exceeds the fully supervised semantic segmentation methods, and achieves high-precision semi-supervised semantic segmentation effectively.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Wang Z, Zhao Z, Xing X, Xu D, Kong X, Zhou L. Conflict-based cross-view consistency for semi-supervised semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2023. pp. 19585–19595. Wang Z, Zhao Z, Xing X, Xu D, Kong X, Zhou L. Conflict-based cross-view consistency for semi-supervised semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2023. pp. 19585–19595.
2.
go back to reference Dutta SK, Saikia S, Barman A, Roy R, Bora K, Mahanta LB, Suresh R. Study on enhanced deep learning approaches for value-added identification and segmentation of striation marks in bullets for precise firearm classification. Appl Soft Comput. 2021;112:107789.CrossRef Dutta SK, Saikia S, Barman A, Roy R, Bora K, Mahanta LB, Suresh R. Study on enhanced deep learning approaches for value-added identification and segmentation of striation marks in bullets for precise firearm classification. Appl Soft Comput. 2021;112:107789.CrossRef
3.
go back to reference Mohajerani S, Saeedi P. Cloud-net: An end-to-end cloud detection algorithm for landsat 8 imagery. In: IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium; 2019. pp. 1029–1032. IEEE. Mohajerani S, Saeedi P. Cloud-net: An end-to-end cloud detection algorithm for landsat 8 imagery. In: IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium; 2019. pp. 1029–1032. IEEE.
4.
go back to reference Yu Q, Gao Y, Zheng Y, Zhu J, Dai Y, Shi Y. Crossover-net: Leveraging vertical-horizontal crossover relation for robust medical image segmentation. Pattern Recogn. 2021;113:107756.CrossRef Yu Q, Gao Y, Zheng Y, Zhu J, Dai Y, Shi Y. Crossover-net: Leveraging vertical-horizontal crossover relation for robust medical image segmentation. Pattern Recogn. 2021;113:107756.CrossRef
5.
go back to reference Mittal S, Tatarchenko M, Brox T. Semi-supervised semantic segmentation with high-and low-level consistency. IEEE Trans Pattern Anal Mach Intell. 2019;43(4):1369–79.CrossRef Mittal S, Tatarchenko M, Brox T. Semi-supervised semantic segmentation with high-and low-level consistency. IEEE Trans Pattern Anal Mach Intell. 2019;43(4):1369–79.CrossRef
6.
go back to reference Sun L, Wu J, Ding X, Huang Y, Wang G, Yu Y. A teacher-student framework for semi-supervised medical image segmentation from mixed supervision. IEEE Trans Med Imaging. 2020. Sun L, Wu J, Ding X, Huang Y, Wang G, Yu Y. A teacher-student framework for semi-supervised medical image segmentation from mixed supervision. IEEE Trans Med Imaging. 2020.
7.
go back to reference Kurakin A, Raffel C, Berthelot D, Cubuk ED, Zhang H, Sohn K, Carlini N. Remixmatch: Semi-supervised learning with distribution matching and augmentation anchoring. ICLR. 2020. Kurakin A, Raffel C, Berthelot D, Cubuk ED, Zhang H, Sohn K, Carlini N. Remixmatch: Semi-supervised learning with distribution matching and augmentation anchoring. ICLR. 2020.
8.
go back to reference Miyato T, Maeda S-I, Koyama M, Ishii S. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans Pattern Anal Mach Intell. 2018;41(8):1979–93.CrossRef Miyato T, Maeda S-I, Koyama M, Ishii S. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans Pattern Anal Mach Intell. 2018;41(8):1979–93.CrossRef
9.
go back to reference Zhang H, Cisse M, Dauphin YN, Lopez-Paz D. Mixup: Beyond empirical risk minimization. ICLR. 2017. Zhang H, Cisse M, Dauphin YN, Lopez-Paz D. Mixup: Beyond empirical risk minimization. ICLR. 2017.
10.
go back to reference Verma V, Kawaguchi K, Lamb A, Kannala J, Solin A, Bengio Y, Lopez-Paz D. Interpolation consistency training for semi-supervised learning. Neural Netw. 2022;145:90–106.CrossRef Verma V, Kawaguchi K, Lamb A, Kannala J, Solin A, Bengio Y, Lopez-Paz D. Interpolation consistency training for semi-supervised learning. Neural Netw. 2022;145:90–106.CrossRef
11.
go back to reference Li X, Yu L, Chen H, Fu C-W, Xing L, Heng P-A. Transformation-consistent self-ensembling model for semisupervised medical image segmentation. IEEE Trans Neural Netw Learn Syst. 2020;32(2):523–34.CrossRef Li X, Yu L, Chen H, Fu C-W, Xing L, Heng P-A. Transformation-consistent self-ensembling model for semisupervised medical image segmentation. IEEE Trans Neural Netw Learn Syst. 2020;32(2):523–34.CrossRef
12.
go back to reference Fang K, Li WJ. Dmnet: difference minimization network for semi-supervised segmentation in medical images. In: International Conference on Medical Image Computing and Computer-Assisted Intervention; 2020. pp. 532–541. Springer. Fang K, Li WJ. Dmnet: difference minimization network for semi-supervised segmentation in medical images. In: International Conference on Medical Image Computing and Computer-Assisted Intervention; 2020. pp. 532–541. Springer.
13.
go back to reference Zheng X, Fu C, Xie H, Chen J, Wang X, Sham C-W. Uncertainty-aware deep co-training for semi-supervised medical image segmentation. Comput Biol Med. 2022;149:106051.CrossRef Zheng X, Fu C, Xie H, Chen J, Wang X, Sham C-W. Uncertainty-aware deep co-training for semi-supervised medical image segmentation. Comput Biol Med. 2022;149:106051.CrossRef
14.
go back to reference Shi Y, Zhang J, Ling T, Lu J, Zheng Y, Yu Q, Qi L, Gao Y. Inconsistency-aware uncertainty estimation for semi-supervised medical image segmentation. IEEE Trans Med Imaging. 2021;41(3):608–20.CrossRef Shi Y, Zhang J, Ling T, Lu J, Zheng Y, Yu Q, Qi L, Gao Y. Inconsistency-aware uncertainty estimation for semi-supervised medical image segmentation. IEEE Trans Med Imaging. 2021;41(3):608–20.CrossRef
15.
go back to reference Kervadec H, Dolz J, Granger É, Ben Ayed I. Curriculum semi-supervised segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention; 2019. pp. 568–576. Springer. Kervadec H, Dolz J, Granger É, Ben Ayed I. Curriculum semi-supervised segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention; 2019. pp. 568–576. Springer.
16.
go back to reference Liu X, Hu G, Ma X, Kuang H. An enhanced neural network based on deep metric learning for skin lesion segmentation. In: 2019 Chinese Control And Decision Conference (CCDC); 2019. pp. 1633–1638. IEEE. Liu X, Hu G, Ma X, Kuang H. An enhanced neural network based on deep metric learning for skin lesion segmentation. In: 2019 Chinese Control And Decision Conference (CCDC); 2019. pp. 1633–1638. IEEE.
17.
go back to reference Luther K, Seung HS. Learning metric graphs for neuron segmentation in electron microscopy images. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019); 2019. pp. 244–248. IEEE. Luther K, Seung HS. Learning metric graphs for neuron segmentation in electron microscopy images. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019); 2019. pp. 244–248. IEEE.
18.
go back to reference Xu Z, Wang Y, Lu D, Yu L, Yan J, Luo J, Ma K, Zheng Y, Tong RKY. All-around real label supervision: Cyclic prototype consistency learning for semi-supervised medical image segmentation. IEEE J Biomed Health Inform. 2022. Xu Z, Wang Y, Lu D, Yu L, Yan J, Luo J, Ma K, Zheng Y, Tong RKY. All-around real label supervision: Cyclic prototype consistency learning for semi-supervised medical image segmentation. IEEE J Biomed Health Inform. 2022.
19.
go back to reference Xiao M, Kortylewski A, Wu R, Qiao S, Shen W, Yuille A. Tdmpnet: Prototype network with recurrent top-down modulation for robust object classification under partial occlusion. In: European Conference on Computer Vision; 2020. pp. 447–463. Springer. Xiao M, Kortylewski A, Wu R, Qiao S, Shen W, Yuille A. Tdmpnet: Prototype network with recurrent top-down modulation for robust object classification under partial occlusion. In: European Conference on Computer Vision; 2020. pp. 447–463. Springer.
20.
go back to reference Xu W, Xian Y, Wang J, Schiele B, Akata Z. Attribute prototype network for zero-shot learning. Adv Neural Inf Process Syst. 2020;33:21969–80. Xu W, Xian Y, Wang J, Schiele B, Akata Z. Attribute prototype network for zero-shot learning. Adv Neural Inf Process Syst. 2020;33:21969–80.
21.
go back to reference Liu Y, Zhang X, Zhang S, He X. Part-aware prototype network for few-shot semantic segmentation. In: European Conference on Computer Vision; 2020. pp. 142–158. Springer. Liu Y, Zhang X, Zhang S, He X. Part-aware prototype network for few-shot semantic segmentation. In: European Conference on Computer Vision; 2020. pp. 142–158. Springer.
22.
go back to reference Li Y, Gupta A. Beyond grids: Learning graph representations for visual recognition. Adv Neural Inf Process. 2018;31. Li Y, Gupta A. Beyond grids: Learning graph representations for visual recognition. Adv Neural Inf Process. 2018;31.
23.
go back to reference Lu Y, Chen Y, Zhao D, Chen J. Graph-fcn for image semantic segmentation. In: International Symposium on Neural Networks; 2019. pp. 97–105. Springer. Lu Y, Chen Y, Zhao D, Chen J. Graph-fcn for image semantic segmentation. In: International Symposium on Neural Networks; 2019. pp. 97–105. Springer.
24.
go back to reference Zhang L, Li X, Arnab A, Yang K, Tong Y, Torr PH. Dual graph convolutional network for semantic segmentation. IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019. Zhang L, Li X, Arnab A, Yang K, Tong Y, Torr PH. Dual graph convolutional network for semantic segmentation. IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.
25.
go back to reference Wan J, Liu Y, Wei D, Bai X, Xu Y. Super-bpd: Super boundary-to-pixel direction for fast image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2020 pp. 9253–9262. Wan J, Liu Y, Wei D, Bai X, Xu Y. Super-bpd: Super boundary-to-pixel direction for fast image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2020 pp. 9253–9262.
26.
go back to reference Suzuki T. Superpixel segmentation via convolutional neural networks with regularized information maximization. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 2020. pp. 2573–2577. IEEE. Suzuki T. Superpixel segmentation via convolutional neural networks with regularized information maximization. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 2020. pp. 2573–2577. IEEE.
27.
go back to reference Wang K, Liew JH, Zou Y, Zhou D, Feng J. Panet: Few-shot image semantic segmentation with prototype alignment. In: Proceedings of the IEEE/CVF International Conference on Computer Vision; 2019. pp. 9197–9206. Wang K, Liew JH, Zou Y, Zhou D, Feng J. Panet: Few-shot image semantic segmentation with prototype alignment. In: Proceedings of the IEEE/CVF International Conference on Computer Vision; 2019. pp. 9197–9206.
28.
go back to reference Zhou J, Cui G, Hu S, Zhang Z, Yang C, Liu Z, Wang L, Li C, Sun M. Graph neural networks: A review of methods and applications. AI Open. 2020;1:57–81.CrossRef Zhou J, Cui G, Hu S, Zhang Z, Yang C, Liu Z, Wang L, Li C, Sun M. Graph neural networks: A review of methods and applications. AI Open. 2020;1:57–81.CrossRef
29.
go back to reference Jaeger S, Karargyris A, Candemir S, Folio L, Siegelman J, Callaghan F, Xue Z, Palaniappan K, Singh RK, Antani S, et al. Automatic tuberculosis screening using chest radiographs. IEEE Trans Med Imaging. 2013;33(2):233–45.CrossRef Jaeger S, Karargyris A, Candemir S, Folio L, Siegelman J, Callaghan F, Xue Z, Palaniappan K, Singh RK, Antani S, et al. Automatic tuberculosis screening using chest radiographs. IEEE Trans Med Imaging. 2013;33(2):233–45.CrossRef
30.
go back to reference Shiraishi J, Katsuragawa S, Ikezoe J, Matsumoto T, Kobayashi T, Komatsu K-I, Matsui M, Fujita H, Kodera Y, Doi K. Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules. Am J Roentgenol. 2000;174(1):71–4.CrossRef Shiraishi J, Katsuragawa S, Ikezoe J, Matsumoto T, Kobayashi T, Komatsu K-I, Matsui M, Fujita H, Kodera Y, Doi K. Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules. Am J Roentgenol. 2000;174(1):71–4.CrossRef
31.
go back to reference Van Ginneken B, Stegmann MB, Loog M. Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database. Med Image Anal. 2006;10(1):19–40.CrossRef Van Ginneken B, Stegmann MB, Loog M. Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database. Med Image Anal. 2006;10(1):19–40.CrossRef
32.
go back to reference Bosdelekidis V, Ioakeimidis NS. Lung field segmentation in chest x-rays: a deformation-tolerant procedure based on the approximation of rib cage seed points. Appl Sci. 2020;10(18):6264.CrossRef Bosdelekidis V, Ioakeimidis NS. Lung field segmentation in chest x-rays: a deformation-tolerant procedure based on the approximation of rib cage seed points. Appl Sci. 2020;10(18):6264.CrossRef
33.
go back to reference Candemir S, Jaeger S, Palaniappan K, Musco JP, Singh RK, Xue Z, Karargyris A, Antani S, Thoma G, McDonald CJ. Lung segmentation in chest radiographs using anatomical atlases with nonrigid registration. IEEE Trans Med Imaging. 2013;33(2):577–90.CrossRef Candemir S, Jaeger S, Palaniappan K, Musco JP, Singh RK, Xue Z, Karargyris A, Antani S, Thoma G, McDonald CJ. Lung segmentation in chest radiographs using anatomical atlases with nonrigid registration. IEEE Trans Med Imaging. 2013;33(2):577–90.CrossRef
34.
go back to reference Oliveira H, dos Santos J. Deep transfer learning for segmentation of anatomical structures in chest radiographs. In: 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI); 2018. pp. 204–211. IEEE. Oliveira H, dos Santos J. Deep transfer learning for segmentation of anatomical structures in chest radiographs. In: 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI); 2018. pp. 204–211. IEEE.
35.
go back to reference Souza JC, Diniz JOB, Ferreira JL, da Silva GLF, Silva AC, de Paiva AC. An automatic method for lung segmentation and reconstruction in chest x-ray using deep neural networks. Comput Methods Programs Biomed. 2019;177:285–96.CrossRef Souza JC, Diniz JOB, Ferreira JL, da Silva GLF, Silva AC, de Paiva AC. An automatic method for lung segmentation and reconstruction in chest x-ray using deep neural networks. Comput Methods Programs Biomed. 2019;177:285–96.CrossRef
36.
go back to reference Wang H, Gu H, Qin P, Wang J. U-shaped gan for semi-supervised learning and unsupervised domain adaptation in high resolution chest radiograph segmentation. Front Med. 2021;8. Wang H, Gu H, Qin P, Wang J. U-shaped gan for semi-supervised learning and unsupervised domain adaptation in high resolution chest radiograph segmentation. Front Med. 2021;8.
37.
go back to reference Scherer S, Schön R, Lienhart R. Pseudo-label noise suppression techniques for semi-supervised semantic segmentation. 2022. arXiv preprint arXiv:2210.10426. Scherer S, Schön R, Lienhart R. Pseudo-label noise suppression techniques for semi-supervised semantic segmentation. 2022. arXiv preprint arXiv:​2210.​10426.
38.
go back to reference Yang L, Zhuo W, Qi L, Shi Y, Gao Y. St++: Make self-training work better for semi-supervised semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2022. pp. 4268–4277. Yang L, Zhuo W, Qi L, Shi Y, Gao Y. St++: Make self-training work better for semi-supervised semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2022. pp. 4268–4277.
39.
go back to reference Xie H, Wang C, Zheng M, Dong M, You S, Fu C, Xu C. Boosting semi-supervised semantic segmentation with probabilistic representations. In: Proceedings of the AAAI Conference on Artificial Intelligence; 2023. vol. 37, pp. 2938–2946. Xie H, Wang C, Zheng M, Dong M, You S, Fu C, Xu C. Boosting semi-supervised semantic segmentation with probabilistic representations. In: Proceedings of the AAAI Conference on Artificial Intelligence; 2023. vol. 37, pp. 2938–2946.
40.
go back to reference Novikov AA, Lenis D, Major D, Hladůvka J, Wimmer M, Bühler K. Fully convolutional architectures for multiclass segmentation in chest radiographs. IEEE Trans Med Imaging. 2018;37(8):1865–76.CrossRef Novikov AA, Lenis D, Major D, Hladůvka J, Wimmer M, Bühler K. Fully convolutional architectures for multiclass segmentation in chest radiographs. IEEE Trans Med Imaging. 2018;37(8):1865–76.CrossRef
41.
go back to reference Dai W, Dong N, Wang Z, Liang X, Zhang H, Xing EP. Scan: Structure correcting adversarial network for organ segmentation in chest x-rays. In: International Workshop on Deep Learning in Medical Image Analysis; 2018. pp. 263–273. Springer. Dai W, Dong N, Wang Z, Liang X, Zhang H, Xing EP. Scan: Structure correcting adversarial network for organ segmentation in chest x-rays. In: International Workshop on Deep Learning in Medical Image Analysis; 2018. pp. 263–273. Springer.
42.
go back to reference Shah MP, Merchant S, Awate SP. Ms-net: mixed-supervision fully-convolutional networks for full-resolution segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention; 2018. pp. 379–387. Springer. Shah MP, Merchant S, Awate SP. Ms-net: mixed-supervision fully-convolutional networks for full-resolution segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention; 2018. pp. 379–387. Springer.
43.
go back to reference Bortsova G, Dubost F, Hogeweg L, Katramados I, Bruijne MD. Semi-supervised medical image segmentation via learning consistency under transformations. In: International Conference on Medical Image Computing and Computer-Assisted Intervention; 2019. pp. 810–818. Springer. Bortsova G, Dubost F, Hogeweg L, Katramados I, Bruijne MD. Semi-supervised medical image segmentation via learning consistency under transformations. In: International Conference on Medical Image Computing and Computer-Assisted Intervention; 2019. pp. 810–818. Springer.
Metadata
Title
Prototype Consistency Learning for Medical Image Segmentation by Cross Pseudo Supervision
Authors
Lu Xie
Weigang Li
Yongqiang Wang
Yuntao Zhao
Publication date
07-09-2023
Publisher
Springer US
Published in
Cognitive Computation / Issue 1/2024
Print ISSN: 1866-9956
Electronic ISSN: 1866-9964
DOI
https://doi.org/10.1007/s12559-023-10198-5

Premium Partner