Skip to main content
Top

2019 | OriginalPaper | Chapter

17. Deep Learning with Convolutional Neural Networks for Histopathology Image Analysis

Authors : Dragan Bošnački, Natal van Riel, Mitko Veta

Published in: Automated Reasoning for Systems Biology and Medicine

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In the recent years, deep learning based methods and, in particular, convolutional neural networks, have been dominating the arena of medical image analysis. This has been made possible both with the advent of new parallel hardware and the development of efficient algorithms. It is expected that future advances in both of these directions will increase this domination. The application of deep learning methods to medical image analysis has been shown to significantly improve the accuracy and efficiency of the diagnoses. In this chapter, we focus on applications of deep learning in microscopy image analysis and digital pathology, in particular. We provide an overview of the state-of-the-art methods in this area and exemplify some of the main techniques. Finally, we discuss some open challenges and avenues for future work.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Footnotes
1
Actually b can be considered as a special weight \(w_0\) associated with a special input \(x_0\) which has a constant value 1. In this way the transfer function becomes slightly simpler \(\sigma (\mathbf {w}^T \mathbf {x})\). However, for the sake of clarity here we keep these two parameters separately.
 
2
Actually this is a definition of a cross-correlation which is slightly different than the usual mathematical notion of convolution, but in the machine learning practice this is how the convolution operation is implemented [23].
 
3
In principle, one can unfold the \(m \times n\) covered rectangular patch of the input and the filter into l-dimensional vectors, where \(l = m \times n\). In this way, “*” becomes real a dot product between the vectors. Also, a bias element can be added, like in the traditional neural networks.
 
4
In recent years, there is growing a trend to use fully convolutional networks in which the fully connected layers are implemented by means of convolutional layers.
 
Literature
8.
go back to reference Bekkers EJ, Lafarge MW, Veta M, Eppenhof KAJ, Pluim JPW, Duits R (2018) Roto-translation covariant convolutional networks for medical image analysis. CoRR. arXiv:1804.03393 Bekkers EJ, Lafarge MW, Veta M, Eppenhof KAJ, Pluim JPW, Duits R (2018) Roto-translation covariant convolutional networks for medical image analysis. CoRR. arXiv:​1804.​03393
9.
go back to reference Christ PF, Ettlinger F, Grün F, Elshaer MEA, Lipková J, Schlecht S, Ahmaddy F, Tatavarty S, Bickel M, Bilic P, Rempfler M, Hofmann F, D’Anastasi M, Ahmadi S, Kaissis G, Holch J, Sommer WH, Braren R, Heinemann V, Menze BH (2017) Automatic liver and tumor segmentation of CT and MRI volumes using cascaded fully convolutional neural networks. CoRR arXiv:1702.05970 Christ PF, Ettlinger F, Grün F, Elshaer MEA, Lipková J, Schlecht S, Ahmaddy F, Tatavarty S, Bickel M, Bilic P, Rempfler M, Hofmann F, D’Anastasi M, Ahmadi S, Kaissis G, Holch J, Sommer WH, Braren R, Heinemann V, Menze BH (2017) Automatic liver and tumor segmentation of CT and MRI volumes using cascaded fully convolutional neural networks. CoRR arXiv:​1702.​05970
11.
go back to reference Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O (2016) 3d u-net: learning dense volumetric segmentation from sparse annotation. CoRR. arXiv:1606.06650 Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O (2016) 3d u-net: learning dense volumetric segmentation from sparse annotation. CoRR. arXiv:​1606.​06650
13.
go back to reference Ciresan DC, Giusti A, Gambardella LM, Schmidhuber J (2013) Mitosis detection in breast cancer histology images with deep neural networks. In: Mori K, Sakuma I, Sato Y, Barillot C, Navab N (eds.) Medical image computing and computer-assisted intervention - MICCAI 2013 - 16th international conference, Nagoya, Japan, September 22-26, 2013, Proceedings, Part II, Lecture Notes in Computer Science, vol 8150. Springer, pp 411–418. https://doi.org/10.1007/978-3-642-40763-5_51CrossRef Ciresan DC, Giusti A, Gambardella LM, Schmidhuber J (2013) Mitosis detection in breast cancer histology images with deep neural networks. In: Mori K, Sakuma I, Sato Y, Barillot C, Navab N (eds.) Medical image computing and computer-assisted intervention - MICCAI 2013 - 16th international conference, Nagoya, Japan, September 22-26, 2013, Proceedings, Part II, Lecture Notes in Computer Science, vol 8150. Springer, pp 411–418. https://​doi.​org/​10.​1007/​978-3-642-40763-5_​51CrossRef
14.
go back to reference Codella NCF, Anderson D, Philips T, Porto A, Massey K, Snowdon J, Feris RS, Smith JR (2018) Segmentation of both diseased and healthy skin from clinical photographs in a primary care setting. CoRR. arXiv:1804.05944 Codella NCF, Anderson D, Philips T, Porto A, Massey K, Snowdon J, Feris RS, Smith JR (2018) Segmentation of both diseased and healthy skin from clinical photographs in a primary care setting. CoRR. arXiv:​1804.​05944
15.
go back to reference Cruz-Roa A, Gilmore H, Basavanhally A, Feldman M, Ganesan S, Shih NNC, Tomaszewski J, González FA, Madabhushi A (2017) Accurate and reproducible invasive breast cancer detection in whole-slide images: a deep learning approach for quantifying tumor extent. Scientif Rep 7:46450 EP. https://doi.org/10.1038/srep46450 Cruz-Roa A, Gilmore H, Basavanhally A, Feldman M, Ganesan S, Shih NNC, Tomaszewski J, González FA, Madabhushi A (2017) Accurate and reproducible invasive breast cancer detection in whole-slide images: a deep learning approach for quantifying tumor extent. Scientif Rep 7:46450 EP. https://​doi.​org/​10.​1038/​srep46450
16.
go back to reference Donahue J, Hendricks LA, Guadarrama S, Rohrbach M, Venugopalan S, Darrell T, Saenko K (2015) Long-term recurrent convolutional networks for visual recognition and description. In: IEEE conference on computer vision and pattern recognition, CVPR 2015, Boston, MA, USA, 7–12 June 2015, vol 2, pp 2625–2634. https://doi.org/10.1109/CVPR.2015.7298878 Donahue J, Hendricks LA, Guadarrama S, Rohrbach M, Venugopalan S, Darrell T, Saenko K (2015) Long-term recurrent convolutional networks for visual recognition and description. In: IEEE conference on computer vision and pattern recognition, CVPR 2015, Boston, MA, USA, 7–12 June 2015, vol 2, pp 2625–2634. https://​doi.​org/​10.​1109/​CVPR.​2015.​7298878
17.
go back to reference Dozat T (2015) Incorporating nesterov momentum into adam Dozat T (2015) Incorporating nesterov momentum into adam
18.
go back to reference Drozdzal M, Vorontsov E, Chartrand G, Kadoury S, Pal C (2016) The importance of skip connections in biomedical image segmentation. CoRR. arXiv:1608.04117 Drozdzal M, Vorontsov E, Chartrand G, Kadoury S, Pal C (2016) The importance of skip connections in biomedical image segmentation. CoRR. arXiv:​1608.​04117
19.
go back to reference Fukushima K (1980) Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybernet 36:193–202CrossRef Fukushima K (1980) Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybernet 36:193–202CrossRef
27.
go back to reference Huang G, Liu Z, van der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017. IEEE Computer Society, pp 2261–2269. https://doi.org/10.1109/CVPR.2017.243 Huang G, Liu Z, van der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017, Honolulu, HI, USA, 21–26 July 2017. IEEE Computer Society, pp 2261–2269. https://​doi.​org/​10.​1109/​CVPR.​2017.​243
29.
go back to reference Katz G, Barrett C, Dill DL, Julian K, Kochenderfer MJ (2017) Towards proving the adversarial robustness of deep neural networks. In: Bulwahn L, Kamali M, Linker S (eds.) Proceedings first workshop on formal verification of autonomous vehicles, FVAV@iFM 2017, Turin, Italy, 19th September 2017. EPTCS, vol 257, pp 19–26. https://doi.org/10.4204/EPTCS.257.3CrossRef Katz G, Barrett C, Dill DL, Julian K, Kochenderfer MJ (2017) Towards proving the adversarial robustness of deep neural networks. In: Bulwahn L, Kamali M, Linker S (eds.) Proceedings first workshop on formal verification of autonomous vehicles, FVAV@iFM 2017, Turin, Italy, 19th September 2017. EPTCS, vol 257, pp 19–26. https://​doi.​org/​10.​4204/​EPTCS.​257.​3CrossRef
32.
go back to reference Lafarge MW, Pluim JPW, Eppenhof KAJ, Moeskops P, Veta M (2017) Domain-adversarial neural networks to address the appearance variability of histopathology images. CoRR. arXiv:1707.06183 Lafarge MW, Pluim JPW, Eppenhof KAJ, Moeskops P, Veta M (2017) Domain-adversarial neural networks to address the appearance variability of histopathology images. CoRR. arXiv:​1707.​06183
34.
go back to reference Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: common objects in context. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) Computer Vision - ECCV 2014. Springer International Publishing, Cham, pp 740–755 Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: common objects in context. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) Computer Vision - ECCV 2014. Springer International Publishing, Cham, pp 740–755
35.
go back to reference Litjens GJS, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Imag Anal 42:60–88CrossRef Litjens GJS, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Imag Anal 42:60–88CrossRef
37.
go back to reference Milletari F, Navab N, Ahmadi S (2016) V-net: fully convolutional neural networks for volumetric medical image segmentation. CoRR arXiv:1606.04797 Milletari F, Navab N, Ahmadi S (2016) V-net: fully convolutional neural networks for volumetric medical image segmentation. CoRR arXiv:​1606.​04797
39.
go back to reference Paeng K, Hwang S, Park S, Kim M (2017) A unified framework for tumor proliferation score prediction in breast histopathology. In: Cardoso MJ, Arbel T, Carneiro G, Syeda-Mahmood TF, Tavares JMRS, Moradi M, Bradley AP, Greenspan H, Papa JP, Madabhushi A, Nascimento JC, Cardoso JS, Belagiannis V, Lu Z (eds.) Deep learning in medical image analysis and multimodal learning for clinical decision support - Third international workshop, DLMIA 2017, and 7th international workshop, ML-CDS 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 14, 2017, Proceedings, Lecture Notes in Computer Science, vol 10553. Springer, pp 231–239. https://doi.org/10.1007/978-3-319-67558-9_27CrossRef Paeng K, Hwang S, Park S, Kim M (2017) A unified framework for tumor proliferation score prediction in breast histopathology. In: Cardoso MJ, Arbel T, Carneiro G, Syeda-Mahmood TF, Tavares JMRS, Moradi M, Bradley AP, Greenspan H, Papa JP, Madabhushi A, Nascimento JC, Cardoso JS, Belagiannis V, Lu Z (eds.) Deep learning in medical image analysis and multimodal learning for clinical decision support - Third international workshop, DLMIA 2017, and 7th international workshop, ML-CDS 2017, Held in Conjunction with MICCAI 2017, Québec City, QC, Canada, September 14, 2017, Proceedings, Lecture Notes in Computer Science, vol 10553. Springer, pp 231–239. https://​doi.​org/​10.​1007/​978-3-319-67558-9_​27CrossRef
40.
42.
go back to reference Shi X, Chen Z, Wang H, Yeung D, Wong W, Woo W (2015) Convolutional LSTM network: a machine learning approach for precipitation nowcasting. CoRR. arXiv:1506.04214 Shi X, Chen Z, Wang H, Yeung D, Wong W, Woo W (2015) Convolutional LSTM network: a machine learning approach for precipitation nowcasting. CoRR. arXiv:​1506.​04214
43.
go back to reference Sundermann B, Feder S, Wersching H, Teuber A, Schwindt W, Kugel H, Heindel W, Arolt V, Berger K, Pfleiderer B (2017) Diagnostic classification of unipolar depression based on resting-state functional connectivity MRI: effects of generalization to a diverse sample. J Neural Trans 124(5):589–605. https://doi.org/10.1007/s00702-016-1673-8CrossRef Sundermann B, Feder S, Wersching H, Teuber A, Schwindt W, Kugel H, Heindel W, Arolt V, Berger K, Pfleiderer B (2017) Diagnostic classification of unipolar depression based on resting-state functional connectivity MRI: effects of generalization to a diverse sample. J Neural Trans 124(5):589–605. https://​doi.​org/​10.​1007/​s00702-016-1673-8CrossRef
44.
go back to reference Szegedy C, Liu W, Jia Y, Sermanet P, Reed SE, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: IEEE conference on computer vision and pattern recognition, CVPR 2015, Boston, MA, USA, June 7–12 2015, pp. 1–9. https://doi.org/10.1109/CVPR.2015.7298594 Szegedy C, Liu W, Jia Y, Sermanet P, Reed SE, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: IEEE conference on computer vision and pattern recognition, CVPR 2015, Boston, MA, USA, June 7–12 2015, pp. 1–9. https://​doi.​org/​10.​1109/​CVPR.​2015.​7298594
45.
go back to reference Trajanovski S, Mavroeidis D, Swisher CL, Gebre BG, Veeling B, Wiemker R, Klinder T, Tahmasebi A, Regis SM, Wald C, McKee BJ, MacMahon H, Pien H (2018) Towards radiologist-level cancer risk assessment in CT lung screening using deep learning. CoRR. arXiv:1804.01901 Trajanovski S, Mavroeidis D, Swisher CL, Gebre BG, Veeling B, Wiemker R, Klinder T, Tahmasebi A, Regis SM, Wald C, McKee BJ, MacMahon H, Pien H (2018) Towards radiologist-level cancer risk assessment in CT lung screening using deep learning. CoRR. arXiv:​1804.​01901
46.
go back to reference Vanschoren J, van Rijn JN, Bischl B (2015) Taking machine learning research online with openml. In: Proceedings of the 4th international workshop on big data, streams and heterogeneous source mining: algorithms, systems, programming models and applications, BigMine 2015, Sydney, Australia, August 10 2015. JMLR Workshop and Conference Proceedings, vol 41, pp 1–4. JMLR.org. http://jmlr.org/proceedings/papers/v41/vanschoren15.html Vanschoren J, van Rijn JN, Bischl B (2015) Taking machine learning research online with openml. In: Proceedings of the 4th international workshop on big data, streams and heterogeneous source mining: algorithms, systems, programming models and applications, BigMine 2015, Sydney, Australia, August 10 2015. JMLR Workshop and Conference Proceedings, vol 41, pp 1–4. JMLR.org. http://​jmlr.​org/​proceedings/​papers/​v41/​vanschoren15.​html
49.
50.
go back to reference Xu Y, Li Y, Liu M, Wang Y, Lai M, Chang EI (2016) Gland instance segmentation by deep multichannel side supervision. CoRR. arXiv:1607.03222 Xu Y, Li Y, Liu M, Wang Y, Lai M, Chang EI (2016) Gland instance segmentation by deep multichannel side supervision. CoRR. arXiv:​1607.​03222
Metadata
Title
Deep Learning with Convolutional Neural Networks for Histopathology Image Analysis
Authors
Dragan Bošnački
Natal van Riel
Mitko Veta
Copyright Year
2019
DOI
https://doi.org/10.1007/978-3-030-17297-8_17

Premium Partner