Skip to main content

2016 | OriginalPaper | Buchkapitel

Fully Convolutional Deep Residual Neural Networks for Brain Tumor Segmentation

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper, a fully convolutional residual neural network (FCR-NN) based on linear identity mappings is implemented for medical image segmentation, employed here in the setting of brain tumors. Inspired by deep residual networks which won the ImageNet ILSVRC 2015 classification challenge, the FCR-NN combines optimization gains from residual identity mappings with a fully convolutional architecture for image segmentation that efficiently accounts for both low- and high-level image features. After training two separate networks, one for the task of whole tumor segmentation and a second for tissue sub-region segmentation, the serial FCR-NN architecture exceeds state-of-the art with complete tumor, core tumor and enhancing tumor validation Dice scores of 0.87, 0.81 and 0.72 respectively. Despite each FCR-NN comprising a complex 22 layer architecture, the fully convolutional design allows for complete segmentation of a tumor volume within 2 s.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks (2012) Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks (2012)
2.
Zurück zum Zitat Simonyan, K., Vedaldi, A., Zisserman, A.: Networks, deep inside convolutional: visualising image classification models and saliency maps. In: ICLR, p. 1 (2014) Simonyan, K., Vedaldi, A., Zisserman, A.: Networks, deep inside convolutional: visualising image classification models and saliency maps. In: ICLR, p. 1 (2014)
3.
Zurück zum Zitat Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 07–12 June, 1–9 Sep 2015 (2015) Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 07–12 June, 1–9 Sep 2015 (2015)
4.
Zurück zum Zitat He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. 7(3), 171–180 (2015). Arxiv.Org He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. 7(3), 171–180 (2015). Arxiv.​Org
5.
Zurück zum Zitat Ciresan, D., Giusti, A.: Deep neural networks segment neuronal membranes in electron microscopy images. In: Advances in Neural Information Processing Systems, pp. 1–9 (2012) Ciresan, D., Giusti, A.: Deep neural networks segment neuronal membranes in electron microscopy images. In: Advances in Neural Information Processing Systems, pp. 1–9 (2012)
6.
Zurück zum Zitat Yang, S., Ramanan, D.: Multi-scale recognition with DAG-CNNs (2015) Yang, S., Ramanan, D.: Multi-scale recognition with DAG-CNNs (2015)
7.
Zurück zum Zitat Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation (2014) Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation (2014)
8.
Zurück zum Zitat Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440 (2014) Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440 (2014)
9.
Zurück zum Zitat Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). doi:10.1007/978-3-319-24574-4_28 CrossRef Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). doi:10.​1007/​978-3-319-24574-4_​28 CrossRef
10.
Zurück zum Zitat Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.-M., Larochelle, H.: Brain tumor segmentation with deep neural networks. In: Proceedings of MICCAI-BRATS (Multimodal Brain Tumor Segmentation Challenge), pp. 29–33 (2015) Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.-M., Larochelle, H.: Brain tumor segmentation with deep neural networks. In: Proceedings of MICCAI-BRATS (Multimodal Brain Tumor Segmentation Challenge), pp. 29–33 (2015)
11.
Zurück zum Zitat Kleesiek, J., Urban, G., Hubert, A., Schwarz, D., Maier-hein, K., Bendszus, M., Biller, A.: NeuroImage deep MRI brain extraction: a 3D convolutional neural network for skull stripping. NeuroImage 129, 460–469 (2016)CrossRef Kleesiek, J., Urban, G., Hubert, A., Schwarz, D., Maier-hein, K., Bendszus, M., Biller, A.: NeuroImage deep MRI brain extraction: a 3D convolutional neural network for skull stripping. NeuroImage 129, 460–469 (2016)CrossRef
12.
Zurück zum Zitat Chen, H., Dou, Q., Yu, L., Heng, P.-A.: VoxResNet: deep voxelwise residual networks for volumetric brain segmentation, pp. 1–9 (2016). arXiv:1608.05895v1 Chen, H., Dou, Q., Yu, L., Heng, P.-A.: VoxResNet: deep voxelwise residual networks for volumetric brain segmentation, pp. 1–9 (2016). arXiv:​1608.​05895v1
13.
Zurück zum Zitat Sachdeva, J., Kumar, V., Gupta, I., Khandelwal, N., Ahuja, C.K.: A novel content-based active contour model for brain tumor segmentation. Magn. Reson. Imaging 30(5), 694–715 (2012)CrossRef Sachdeva, J., Kumar, V., Gupta, I., Khandelwal, N., Ahuja, C.K.: A novel content-based active contour model for brain tumor segmentation. Magn. Reson. Imaging 30(5), 694–715 (2012)CrossRef
14.
Zurück zum Zitat Harati, V., Khayati, R., Farzan, A.: Fully automated tumor segmentation based on improved fuzzy connectedness algorithm in brain MR images. Comput. Biol. Med. 41(7), 483–492 (2011)CrossRef Harati, V., Khayati, R., Farzan, A.: Fully automated tumor segmentation based on improved fuzzy connectedness algorithm in brain MR images. Comput. Biol. Med. 41(7), 483–492 (2011)CrossRef
15.
Zurück zum Zitat Prastawa, M., Bullitt, E., Gerig, G.: Simulation of brain tumors in MR images for evaluation of segmentation efficacy. Med. Image Anal. 13(2), 297–311 (2009)CrossRef Prastawa, M., Bullitt, E., Gerig, G.: Simulation of brain tumors in MR images for evaluation of segmentation efficacy. Med. Image Anal. 13(2), 297–311 (2009)CrossRef
16.
Zurück zum Zitat Menze, B.H., Leemput, K., Lashkari, D., Weber, M.-A., Ayache, N., Golland, P.: A generative model for brain tumor segmentation in multi-modal images. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010. LNCS, vol. 6362, pp. 151–159. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15745-5_19 CrossRef Menze, B.H., Leemput, K., Lashkari, D., Weber, M.-A., Ayache, N., Golland, P.: A generative model for brain tumor segmentation in multi-modal images. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010. LNCS, vol. 6362, pp. 151–159. Springer, Heidelberg (2010). doi:10.​1007/​978-3-642-15745-5_​19 CrossRef
17.
Zurück zum Zitat Hamamci, A., Kucuk, N., Karaman, K., Engin, K., Unal, G.: Tumor - cut: segmentation of brain tumors on contrast enhanced MR images for radiosurgery applications. IEEE Trans. Med. Imaging 31(3), 790–804 (2012) Hamamci, A., Kucuk, N., Karaman, K., Engin, K., Unal, G.: Tumor - cut: segmentation of brain tumors on contrast enhanced MR images for radiosurgery applications. IEEE Trans. Med. Imaging 31(3), 790–804 (2012)
18.
Zurück zum Zitat Zhu, Y., Young, G.S., Xue, Z., Huang, R.Y., You, H., Setayesh, K., Hatabu, H., Cao, F., Wong, S.T.: Semi-automatic segmentation software for quantitative clinical brain glioblastoma evaluation. Acad. Radiol. 19(8), 977–85 (2012) Zhu, Y., Young, G.S., Xue, Z., Huang, R.Y., You, H., Setayesh, K., Hatabu, H., Cao, F., Wong, S.T.: Semi-automatic segmentation software for quantitative clinical brain glioblastoma evaluation. Acad. Radiol. 19(8), 977–85 (2012)
19.
Zurück zum Zitat Menze, B.H., Geremia, E., Ayache, N., Szekely, G.: Segmenting glioma in multi-modal images using a generative-discriminative model for brain lesion segmentation. In: Proceedings of MICCAI-BRATS (Multimodal Brain Tumor Segmentation Challenge), p. 7 (2012) Menze, B.H., Geremia, E., Ayache, N., Szekely, G.: Segmenting glioma in multi-modal images using a generative-discriminative model for brain lesion segmentation. In: Proceedings of MICCAI-BRATS (Multimodal Brain Tumor Segmentation Challenge), p. 7 (2012)
20.
Zurück zum Zitat Meier, R., Reyes, M., Bauer, S., Slotboom, J., Wiest, R.: A hybrid model for multimodal brain tumor segmentation. In: Proceedings of NCI-MICCAI BRATS (Multimodal Brain Tumor Segmentation Challenge), pp. 31–37 (2013) Meier, R., Reyes, M., Bauer, S., Slotboom, J., Wiest, R.: A hybrid model for multimodal brain tumor segmentation. In: Proceedings of NCI-MICCAI BRATS (Multimodal Brain Tumor Segmentation Challenge), pp. 31–37 (2013)
21.
Zurück zum Zitat Bakas, S., Zeng, K., Sotiras, A., Rathore, S., Akbari, H., Gaonkar, B., Rozycki, M., Pati, S., Davatzikos, C.: Segmentation of gliomas in multimodal magnetic resonance imaging volumes based on a hybrid generative - discriminative framework. In: Proceedings of MICCAI-BRATS (Multimodal Brain Tumor Segmentation Challenge), pp. 5–12 (2015) Bakas, S., Zeng, K., Sotiras, A., Rathore, S., Akbari, H., Gaonkar, B., Rozycki, M., Pati, S., Davatzikos, C.: Segmentation of gliomas in multimodal magnetic resonance imaging volumes based on a hybrid generative - discriminative framework. In: Proceedings of MICCAI-BRATS (Multimodal Brain Tumor Segmentation Challenge), pp. 5–12 (2015)
22.
Zurück zum Zitat Menze, B.H., Van Leemput, K., Lashkari, D., Riklin-Raviv, T., Geremia, E., Alberts, E., Gruber, P., Wegener, S., Weber, M.-A., Szekely, G., Ayache, N., Golland, P.: A generative probabilistic model and discriminative extensions for brain lesion segmentation with application to tumor and stroke. IEEE Trans. Med. Imaging 35(4), 933–946 (2016)CrossRef Menze, B.H., Van Leemput, K., Lashkari, D., Riklin-Raviv, T., Geremia, E., Alberts, E., Gruber, P., Wegener, S., Weber, M.-A., Szekely, G., Ayache, N., Golland, P.: A generative probabilistic model and discriminative extensions for brain lesion segmentation with application to tumor and stroke. IEEE Trans. Med. Imaging 35(4), 933–946 (2016)CrossRef
23.
Zurück zum Zitat Bakas, S., et al.: GLISTRboost: combining multimodal MRI segmentation, registration, and biophysical tumor growth modeling with gradient boosting machines for glioma segmentation. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Handels, H. (eds.) BrainLes 2015. LNCS, vol. 9556, pp. 144–155. Springer, Cham (2016). doi:10.1007/978-3-319-30858-6_13 CrossRef Bakas, S., et al.: GLISTRboost: combining multimodal MRI segmentation, registration, and biophysical tumor growth modeling with gradient boosting machines for glioma segmentation. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Handels, H. (eds.) BrainLes 2015. LNCS, vol. 9556, pp. 144–155. Springer, Cham (2016). doi:10.​1007/​978-3-319-30858-6_​13 CrossRef
24.
Zurück zum Zitat Zeng, J., See, A.P., Phallen, J., Jackson, C.M., Belcaid, Z., Ruzevick, J., Durham, N., Meyer, C., Harris, T.J., Albesiano, E., Pradilla, G., Ford, E., Wong, J., Hammers, H.-J., Mathios, D., Tyler, B., Brem, H., Tran, P.T., Pardoll, D., Drake, C.G., Lim, M.: Anti-PD-1 blockade and stereotactic radiation produce long-term survival in mice with intracranial gliomas. Int. J. Rad. Oncol. Biol. Phys. 86(2), 343–349 (2013) Zeng, J., See, A.P., Phallen, J., Jackson, C.M., Belcaid, Z., Ruzevick, J., Durham, N., Meyer, C., Harris, T.J., Albesiano, E., Pradilla, G., Ford, E., Wong, J., Hammers, H.-J., Mathios, D., Tyler, B., Brem, H., Tran, P.T., Pardoll, D., Drake, C.G., Lim, M.: Anti-PD-1 blockade and stereotactic radiation produce long-term survival in mice with intracranial gliomas. Int. J. Rad. Oncol. Biol. Phys. 86(2), 343–349 (2013)
25.
Zurück zum Zitat LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)CrossRef LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)CrossRef
26.
Zurück zum Zitat Pereira, S., Pinto, A., Alves, V., Silva, C.A.: Deep convolutional neural networks for the segmentation of gliomas in multi-sequence MRI. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Handels, H. (eds.) BrainLes 2015. LNCS, vol. 9556, pp. 131–143. Springer, Cham (2016). doi:10.1007/978-3-319-30858-6_12 CrossRef Pereira, S., Pinto, A., Alves, V., Silva, C.A.: Deep convolutional neural networks for the segmentation of gliomas in multi-sequence MRI. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Handels, H. (eds.) BrainLes 2015. LNCS, vol. 9556, pp. 131–143. Springer, Cham (2016). doi:10.​1007/​978-3-319-30858-6_​12 CrossRef
27.
Zurück zum Zitat Lecun, Y., Bengio, Y.: Convolutional networks for images, speech, and time-series (1995) Lecun, Y., Bengio, Y.: Convolutional networks for images, speech, and time-series (1995)
28.
Zurück zum Zitat Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines (2010) Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines (2010)
29.
Zurück zum Zitat Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift, pp. 1–11 (2015). arXiv:1502.03167 Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift, pp. 1–11 (2015). arXiv:​1502.​03167
30.
Zurück zum Zitat Bengio, Y., Boulanger-Lewandowski, N., Pascanu, R.: Advances in optimizing recurrent networks. In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, pp. 8624–8628 (2013) Bengio, Y., Boulanger-Lewandowski, N., Pascanu, R.: Advances in optimizing recurrent networks. In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, pp. 8624–8628 (2013)
31.
Zurück zum Zitat He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification (2015) He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification (2015)
32.
Zurück zum Zitat Mandic, D.P.: A generalized normalized gradient descent algorithm. IEEE Signal Process. Lett. 11(2), 115–118 (2004)CrossRef Mandic, D.P.: A generalized normalized gradient descent algorithm. IEEE Signal Process. Lett. 11(2), 115–118 (2004)CrossRef
33.
Zurück zum Zitat Vedaldi, A., Lenc, K.: MatConvNet. In: Proceedings of the 23rd ACM International Conference on Multimedia - MM 2015, pp. 689–692 (2015) Vedaldi, A., Lenc, K.: MatConvNet. In: Proceedings of the 23rd ACM International Conference on Multimedia - MM 2015, pp. 689–692 (2015)
34.
Zurück zum Zitat Agn, M., Puonti, O., Rosenschöld, P.M., Law, I., Leemput, K.: Brain tumor segmentation using a generative model with an RBM prior on tumor shape. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Handels, H. (eds.) BrainLes 2015. LNCS, vol. 9556, pp. 168–180. Springer, Cham (2016). doi:10.1007/978-3-319-30858-6_15 CrossRef Agn, M., Puonti, O., Rosenschöld, P.M., Law, I., Leemput, K.: Brain tumor segmentation using a generative model with an RBM prior on tumor shape. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Handels, H. (eds.) BrainLes 2015. LNCS, vol. 9556, pp. 168–180. Springer, Cham (2016). doi:10.​1007/​978-3-319-30858-6_​15 CrossRef
35.
Zurück zum Zitat Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., Lanczi, L., Gerstner, E., Weber, M.A., Arbel, T., Avants, B.B., Ayache, N., Buendia, P., Collins, D.L., Cordier, N., Corso, J.J., Criminisi, A., Das, T., Delingette, H., Demiralp, Ç., Durst, C.R., Dojat, M., Doyle, S., Festa, J., Forbes, F., Geremia, E., Glocker, B., Golland, P., Guo, X., Hamamci, A., Iftekharuddin, K.M., Jena, R., John, N.M., Konukoglu, E., Lashkari, D., Mariz, J.A., Meier, R., Pereira, S., Precup, D., Price, S.J., Raviv, T.R., Reza, S.M.S., Ryan, M., Sarikaya, D., Schwartz, L., Shin, H.C., Shotton, J., Silva, C.A., Sousa, N., Subbanna, N.K., Szekely, G., Taylor, T.J., Thomas, O.M., Tustison, N.J., Unal, G., Vasseur, F., Wintermark, M., Ye, D.H., Zhao, L., Zhao, B., Zikic, D., Prastawa, M., Reyes, M., Leemput, K.V.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015) Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., Lanczi, L., Gerstner, E., Weber, M.A., Arbel, T., Avants, B.B., Ayache, N., Buendia, P., Collins, D.L., Cordier, N., Corso, J.J., Criminisi, A., Das, T., Delingette, H., Demiralp, Ç., Durst, C.R., Dojat, M., Doyle, S., Festa, J., Forbes, F., Geremia, E., Glocker, B., Golland, P., Guo, X., Hamamci, A., Iftekharuddin, K.M., Jena, R., John, N.M., Konukoglu, E., Lashkari, D., Mariz, J.A., Meier, R., Pereira, S., Precup, D., Price, S.J., Raviv, T.R., Reza, S.M.S., Ryan, M., Sarikaya, D., Schwartz, L., Shin, H.C., Shotton, J., Silva, C.A., Sousa, N., Subbanna, N.K., Szekely, G., Taylor, T.J., Thomas, O.M., Tustison, N.J., Unal, G., Vasseur, F., Wintermark, M., Ye, D.H., Zhao, L., Zhao, B., Zikic, D., Prastawa, M., Reyes, M., Leemput, K.V.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015)
Metadaten
Titel
Fully Convolutional Deep Residual Neural Networks for Brain Tumor Segmentation
verfasst von
Peter D. Chang
Copyright-Jahr
2016
DOI
https://doi.org/10.1007/978-3-319-55524-9_11