Skip to main content
Top
Published in: International Journal of Computer Assisted Radiology and Surgery 2/2022

08-12-2021 | Original Article

Transfer learning for automatic joint segmentation of thyroid and breast lesions from ultrasound images

Authors: Jinlian Ma, Lingyun Bao, Qiong Lou, Dexing Kong

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 2/2022

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Purpose

It plays a significant role to accurately and automatically segment lesions from ultrasound (US) images in clinical application. Nevertheless, it is extremely challenging because distinct components of heterogeneous lesions are similar to background in US images. In our study, a transfer learning-based method is developed for full-automatic joint segmentation of nodular lesions.

Methods

Transfer learning is a widely used method to build high performing computer vision models. Our transfer learning model is a novel type of densely connected convolutional network (SDenseNet). Specifically, we pre-train SDenseNet based on ImageNet dataset. Then our SDenseNet is designed as a multi-channel model (denoted Mul-DenseNet) for automatically jointly segmenting lesions. As comparison, our SDenseNet using different transfer learning is applied to segmenting nodules, respectively. In our study, we find that more datasets for pre-training and multiple pre-training do not always work in segmentation of nodules, and the performance of transfer learning depends on a judicious choice of dataset and characteristics of targets.

Results

Experimental results illustrate a significant performance of the Mul-DenseNet compared to that of other methods in the study. Specially, for thyroid nodule segmentation, overlap metric (OM), dice ratio (DR), true-positive rate (TPR), false-positive rate (FPR) and modified Hausdorff distance (MHD) are \(0.9257\pm 0.0027\), \(0.9596\pm 0.0009\), \(0.9869\pm 0.0008\), \(0.0183\pm 0.0066\) and \(0.3897\pm 0.3488\) mm, respectively; for breast nodule segmentation, OM, DR, TPR, FPR and MHD are \(0.8912\pm 0.0072\), \(0.9513\pm 0.0038\), \(0.9835\pm 0.0015\), \(0.2381\pm 0.1301\) and \(0.2017\pm 0.0302\) mm, respectively.

Conclusions

The experimental results illustrate our transfer learning models are very effective in segmentation of lesions, which also demonstrate that it is potential of our proposed Mul-DenseNet model in clinical applications. This model can reduce heavy workload of the physicians so that it can avoid misdiagnosis cases due to excessive fatigue. Moreover, it is easy and reproducible to detect lesions without medical expertise.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Chang CY, Huang HC, Chen SJ (2010) Automatic thyroid nodule segmentation and component analysis in ultrasound images. Biomed Eng: Appl, Basis Commun 22(02):81–89 Chang CY, Huang HC, Chen SJ (2010) Automatic thyroid nodule segmentation and component analysis in ultrasound images. Biomed Eng: Appl, Basis Commun 22(02):81–89
2.
go back to reference Chang H, Han J, Zhong C, Snijders AM, Mao JH (2017) Unsupervised transfer learning via multi-scale convolutional sparse coding for biomedical applications. IEEE Trans Pattern Anal Mach Intell 40(5):1182–1194CrossRef Chang H, Han J, Zhong C, Snijders AM, Mao JH (2017) Unsupervised transfer learning via multi-scale convolutional sparse coding for biomedical applications. IEEE Trans Pattern Anal Mach Intell 40(5):1182–1194CrossRef
3.
go back to reference Cheng JZ, Ni D, Chou YH, Qin J, Tiu CM, Chang YC, Huang CS, Shen D, Chen CM (2016) Computer-aided diagnosis with deep learning architecture: applications to breast lesions in us images and pulmonary nodules in ct scans. Sci Rep 6(1):1–13CrossRef Cheng JZ, Ni D, Chou YH, Qin J, Tiu CM, Chang YC, Huang CS, Shen D, Chen CM (2016) Computer-aided diagnosis with deep learning architecture: applications to breast lesions in us images and pulmonary nodules in ct scans. Sci Rep 6(1):1–13CrossRef
4.
go back to reference Ciresan D, Meier U, Schmidhuber J (2012) Multi-column deep neural networks for image classification. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3642–3649. IEEE Ciresan D, Meier U, Schmidhuber J (2012) Multi-column deep neural networks for image classification. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3642–3649. IEEE
5.
go back to reference Faisal A, Ng SC, Goh SL, George J, Supriyanto E, Lai KW (2015) Multiple lrek active contours for knee meniscus ultrasound image segmentation. IEEE Trans Med Imaging 34(10):2162–2171CrossRef Faisal A, Ng SC, Goh SL, George J, Supriyanto E, Lai KW (2015) Multiple lrek active contours for knee meniscus ultrasound image segmentation. IEEE Trans Med Imaging 34(10):2162–2171CrossRef
6.
go back to reference Fang L, Qiu T, Liu Y, Chen C (2018) Active contour model driven by global and local intensity information for ultrasound image segmentation. Comput Math Appl 75(12):4286–4299CrossRef Fang L, Qiu T, Liu Y, Chen C (2018) Active contour model driven by global and local intensity information for ultrasound image segmentation. Comput Math Appl 75(12):4286–4299CrossRef
7.
go back to reference Geert L, Thijs K, Babak EB, Arnaud AAS, Francesco C, Mohsen G, Jeroen ADL, Bram VG, Clara I (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88CrossRef Geert L, Thijs K, Babak EB, Arnaud AAS, Francesco C, Mohsen G, Jeroen ADL, Bram VG, Clara I (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88CrossRef
8.
go back to reference Huang G, Liu Z, Maaten LVD, Weinberger KQ (2017) Densely connected convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2261–2269 Huang G, Liu Z, Maaten LVD, Weinberger KQ (2017) Densely connected convolutional networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 2261–2269
9.
go back to reference Illanes A, Esmaeili N, Poudel P, Balakrishnan S, Friebe M (2019) Parametrical modelling for texture characterization-a novel approach applied to ultrasound thyroid segmentation. PLoS ONE 14(1):e0211215CrossRef Illanes A, Esmaeili N, Poudel P, Balakrishnan S, Friebe M (2019) Parametrical modelling for texture characterization-a novel approach applied to ultrasound thyroid segmentation. PLoS ONE 14(1):e0211215CrossRef
10.
go back to reference Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning, pp 448–456. PMLR Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning, pp 448–456. PMLR
11.
go back to reference Kim S, Choi J, Kim T, Kim C (2019) Self-training and adversarial background regularization for unsupervised domain adaptive one-stage object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 6092–6101 Kim S, Choi J, Kim T, Kim C (2019) Self-training and adversarial background regularization for unsupervised domain adaptive one-stage object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 6092–6101
12.
go back to reference Koundal D, Gupta S, Singh S (2016) Automated delineation of thyroid nodules in ultrasound images using spatial neutrosophic clustering and level set. Appl Soft Comput 40:86–97CrossRef Koundal D, Gupta S, Singh S (2016) Automated delineation of thyroid nodules in ultrasound images using spatial neutrosophic clustering and level set. Appl Soft Comput 40:86–97CrossRef
13.
go back to reference Krizhevsky A, Sutskever I, Hinton G.E (2012) Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp 1097–1105 Krizhevsky A, Sutskever I, Hinton G.E (2012) Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp 1097–1105
14.
go back to reference Long M, Wang J, Ding G, Shen D, Yang Q (2014) Transfer learning with graph co-regularization. IEEE Trans Knowl Data Eng 26(7):1805–1818CrossRef Long M, Wang J, Ding G, Shen D, Yang Q (2014) Transfer learning with graph co-regularization. IEEE Trans Knowl Data Eng 26(7):1805–1818CrossRef
15.
go back to reference Ma J, Kong D (2018) Deep learning models for segmentation of lesion based on ultrasound images. Adv Ultrasound Diagn Therapy 2(2):82–93CrossRef Ma J, Kong D (2018) Deep learning models for segmentation of lesion based on ultrasound images. Adv Ultrasound Diagn Therapy 2(2):82–93CrossRef
16.
go back to reference Ma J, Wu F, Zhao Q, Kong D (2017) Ultrasound image-based thyroid nodule automatic segmentation using convolutional neural networks. Int J Comput Assist Radiol Surg 12(11):1895–1910CrossRef Ma J, Wu F, Zhao Q, Kong D (2017) Ultrasound image-based thyroid nodule automatic segmentation using convolutional neural networks. Int J Comput Assist Radiol Surg 12(11):1895–1910CrossRef
17.
go back to reference Ma J, Wu F, Zhu J, Kong D (2017) Cascade convolutional neural networks for automatic detection of thyroid nodules in ultrasound images. Med Phys 44(5):1678–1691CrossRef Ma J, Wu F, Zhu J, Kong D (2017) Cascade convolutional neural networks for automatic detection of thyroid nodules in ultrasound images. Med Phys 44(5):1678–1691CrossRef
18.
go back to reference Ma J, Wu F, Zhu J, Xu D, Kong D (2017) A pre-trained convolutional neural network based method for thyroid nodule diagnosis. Ultrasonics 73:221–230CrossRef Ma J, Wu F, Zhu J, Xu D, Kong D (2017) A pre-trained convolutional neural network based method for thyroid nodule diagnosis. Ultrasonics 73:221–230CrossRef
19.
go back to reference Ma X, Niu Y, Gu L, Wang Y, Zhao Y, Bailey J, Lu F (2020) Understanding adversarial attacks on deep learning based medical image analysis systems. Pattern Recognition p 107332 Ma X, Niu Y, Gu L, Wang Y, Zhao Y, Bailey J, Lu F (2020) Understanding adversarial attacks on deep learning based medical image analysis systems. Pattern Recognition p 107332
20.
go back to reference Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In: ICML Workshop on Deep Learning for Audio, Speech and Language Processing Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In: ICML Workshop on Deep Learning for Audio, Speech and Language Processing
21.
go back to reference Panigrahi L, Verma K, Singh BK (2019) Ultrasound image segmentation using a novel multi-scale gaussian kernel fuzzy clustering and multi-scale vector field convolution. Expert Syst Appl 115:486–498CrossRef Panigrahi L, Verma K, Singh BK (2019) Ultrasound image segmentation using a novel multi-scale gaussian kernel fuzzy clustering and multi-scale vector field convolution. Expert Syst Appl 115:486–498CrossRef
22.
go back to reference Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M (2014) Imagenet large scale visual recognition challenge. International Journal of Computer Vision pp 1–42 Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M (2014) Imagenet large scale visual recognition challenge. International Journal of Computer Vision pp 1–42
23.
go back to reference Schneider RJ, Perrin DP, Vasilyev NV, Marx GR, Pedro J, Howe RD (2010) Mitral annulus segmentation from 3d ultrasound using graph cuts. IEEE Trans Med Imaging 29(9):1676–1687CrossRef Schneider RJ, Perrin DP, Vasilyev NV, Marx GR, Pedro J, Howe RD (2010) Mitral annulus segmentation from 3d ultrasound using graph cuts. IEEE Trans Med Imaging 29(9):1676–1687CrossRef
24.
go back to reference Seo H, Yu L, Ren H, Li X, Shen L, Xing L (2021) Deep neural network with consistency regularization of multi-output channels for improved tumor detection and delineation. IEEE Transactions on Medical Imaging Seo H, Yu L, Ren H, Li X, Shen L, Xing L (2021) Deep neural network with consistency regularization of multi-output channels for improved tumor detection and delineation. IEEE Transactions on Medical Imaging
25.
go back to reference Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298CrossRef Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298CrossRef
26.
go back to reference Tatiana T, Francesco O, Barbara C (2014) Learning categories from few examples with multi model knowledge transfer. IEEE Trans Pattern Anal Mach Intell 36(5):928–941CrossRef Tatiana T, Francesco O, Barbara C (2014) Learning categories from few examples with multi model knowledge transfer. IEEE Trans Pattern Anal Mach Intell 36(5):928–941CrossRef
27.
go back to reference Wan L, Zeiler M, Zhang S, Cun Y.L, Fergus R (2013) Regularization of neural networks using dropconnect. In: Proceedings of the 30th International Conference on Machine Learning (ICML-13), pp. 1058–1066 Wan L, Zeiler M, Zhang S, Cun Y.L, Fergus R (2013) Regularization of neural networks using dropconnect. In: Proceedings of the 30th International Conference on Machine Learning (ICML-13), pp. 1058–1066
28.
go back to reference Xian M, Zhang Y, Cheng HD (2015) Fully automatic segmentation of breast ultrasound images based on breast characteristics in space and frequency domains. Pattern Recogn 48(2):485-497CrossRef Xian M, Zhang Y, Cheng HD (2015) Fully automatic segmentation of breast ultrasound images based on breast characteristics in space and frequency domains. Pattern Recogn 48(2):485-497CrossRef
29.
go back to reference Xu Y, Fang X, Wu J, Li X, Zhang D (2015) Discriminative transfer subspace learning via low-rank and sparse representation. IEEE Trans Image Process 25(2):850–863CrossRef Xu Y, Fang X, Wu J, Li X, Zhang D (2015) Discriminative transfer subspace learning via low-rank and sparse representation. IEEE Trans Image Process 25(2):850–863CrossRef
30.
go back to reference Xu Y, Pan SJ, Xiong H, Wu Q, Luo R, Min H, Song H (2017) A unified framework for metric transfer learning. IEEE Trans Knowl Data Eng 29(6):1158–1171CrossRef Xu Y, Pan SJ, Xiong H, Wu Q, Luo R, Min H, Song H (2017) A unified framework for metric transfer learning. IEEE Trans Knowl Data Eng 29(6):1158–1171CrossRef
31.
go back to reference Xu Y, Wang Y, Yuan J, Cheng Q, Wang X, Carson PL (2019) Medical breast ultrasound image segmentation by machine learning. Ultrasonics 91:1–9CrossRef Xu Y, Wang Y, Yuan J, Cheng Q, Wang X, Carson PL (2019) Medical breast ultrasound image segmentation by machine learning. Ultrasonics 91:1–9CrossRef
32.
go back to reference Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, pp 3320–3328 Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, pp 3320–3328
33.
go back to reference Zang X, Bascom R, Gilbert C, Toth J, Higgins W (2015) Methods for 2-d and 3-d endobronchial ultrasound image segmentation. IEEE Trans Biomed Eng 63(7):1426–1439CrossRef Zang X, Bascom R, Gilbert C, Toth J, Higgins W (2015) Methods for 2-d and 3-d endobronchial ultrasound image segmentation. IEEE Trans Biomed Eng 63(7):1426–1439CrossRef
34.
go back to reference Zhang L, Zuo W, Zhang D (2016) Lsdt: Latent sparse domain transfer learning for visual adaptation. IEEE Trans Image Process 25(3):1177–1191CrossRef Zhang L, Zuo W, Zhang D (2016) Lsdt: Latent sparse domain transfer learning for visual adaptation. IEEE Trans Image Process 25(3):1177–1191CrossRef
35.
go back to reference Zhang W, Li R, Deng H, Wang L, Lin W, Ji S, Shen D (2015) Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. Neuroimage 108:214–224CrossRef Zhang W, Li R, Deng H, Wang L, Lin W, Ji S, Shen D (2015) Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. Neuroimage 108:214–224CrossRef
Metadata
Title
Transfer learning for automatic joint segmentation of thyroid and breast lesions from ultrasound images
Authors
Jinlian Ma
Lingyun Bao
Qiong Lou
Dexing Kong
Publication date
08-12-2021
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 2/2022
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-021-02505-y

Other articles of this Issue 2/2022

International Journal of Computer Assisted Radiology and Surgery 2/2022 Go to the issue

Premium Partner