Skip to main content
Top
Published in: Neural Computing and Applications 13/2021

10-03-2021 | S.I. : DICTA 2019

Data augmentation for patch-based OCT chorio-retinal segmentation using generative adversarial networks

Authors: Jason Kugelman, David Alonso-Caneiro, Scott A. Read, Stephen J. Vincent, Fred K. Chen, Michael J. Collins

Published in: Neural Computing and Applications | Issue 13/2021

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Many clinical and research tasks rely critically upon the segmentation of tissue layers in optical coherence tomography (OCT) images of the posterior eye (the retina and choroid). However, a major limitation of using machine learning-based segmentation methods is that performance depends on a large quantity of and diversity in the data used to train the models. Due to their demonstrated ability to generate high quality and diverse synthetic images, we propose the application of GANs here to augment data for a patch-based approach to OCT chorio-retinal boundary segmentation. Given the complexity of GAN training, a range of experiments are performed to understand and optimise performance. We show that it is feasible to generate patches that are visually indistinguishable from their real variants and in the best case, the segmentation performance utilising solely synthetic data is nearly comparable to a model trained on real data. The data augmentation capabilities are demonstrated with classification performance improvements realised on a range of sparse datasets. These findings highlight the potential use of GANs for data augmentation in future work with chorio-retinal OCT images. Additionally, this study includes a range of experimental findings and an analysis of techniques which may be useful for developing or improving GAN-based methods that are not necessarily limited to chorio-retinal images, the OCT modality, or data augmentation.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Manjunath V, Goren J, Fujimoto JG, Duker JS (2011) Analysis of choroidal thickness in age-related macular degeneration using spectral-domain optical coherence tomography. Am J Ophthalmol 152(4):663–668CrossRef Manjunath V, Goren J, Fujimoto JG, Duker JS (2011) Analysis of choroidal thickness in age-related macular degeneration using spectral-domain optical coherence tomography. Am J Ophthalmol 152(4):663–668CrossRef
2.
go back to reference Esmaeelpour M et al (2011) Mapping choroidal and retinal thickness variation in type 2 diabetes using three-dimensional 1060-nm optical coherence tomography. Invest Ophthalmol Vis Sci 52(8):5311–5316CrossRef Esmaeelpour M et al (2011) Mapping choroidal and retinal thickness variation in type 2 diabetes using three-dimensional 1060-nm optical coherence tomography. Invest Ophthalmol Vis Sci 52(8):5311–5316CrossRef
3.
go back to reference Sim DA et al (2013) Repeatability and reproducibility of choroidal vessel layer measurements in diabetic retinopathy using enhanced depth optical coherence tomography. Invest Ophthalmol Vis Sci 54(4):2893–2901CrossRef Sim DA et al (2013) Repeatability and reproducibility of choroidal vessel layer measurements in diabetic retinopathy using enhanced depth optical coherence tomography. Invest Ophthalmol Vis Sci 54(4):2893–2901CrossRef
4.
go back to reference Koozekanani D, Boyer K, Roberts C (2001) Retinal thickness measurements from optical coherence tomography using a Markov boundary model. IEEE Trans Med Imaging 20(9):900–916CrossRef Koozekanani D, Boyer K, Roberts C (2001) Retinal thickness measurements from optical coherence tomography using a Markov boundary model. IEEE Trans Med Imaging 20(9):900–916CrossRef
5.
go back to reference Oliveira J, Pereira S, Gonçalves L, Ferreira M, Silva CA (2017) Multi-surface segmentation of OCT images with AMD using sparse high order potentials. Biomed Opt Express 8(1):281–297CrossRef Oliveira J, Pereira S, Gonçalves L, Ferreira M, Silva CA (2017) Multi-surface segmentation of OCT images with AMD using sparse high order potentials. Biomed Opt Express 8(1):281–297CrossRef
6.
go back to reference Cabrera Fernández D, Salinas HM, Puliafito CA (2005) Automated detection of retinal layer structures on optical coherence tomography images. Opt Express 13(25):10200–10216CrossRef Cabrera Fernández D, Salinas HM, Puliafito CA (2005) Automated detection of retinal layer structures on optical coherence tomography images. Opt Express 13(25):10200–10216CrossRef
7.
go back to reference Chiu SJ et al (2010) Automatic segmentation of seven retinal layers in SDOCT images congruent with expert manual segmentation. Opt Express 18(18):19413–19428CrossRef Chiu SJ et al (2010) Automatic segmentation of seven retinal layers in SDOCT images congruent with expert manual segmentation. Opt Express 18(18):19413–19428CrossRef
8.
go back to reference Niu S, de Sisternes L, Chen Q, Leng T, Rubin DL (2016) Automated geographic atrophy segmentation for SD-OCT images using region-based C-V model via local similarity factor. Biomed Opt Express 7(2):581–600CrossRef Niu S, de Sisternes L, Chen Q, Leng T, Rubin DL (2016) Automated geographic atrophy segmentation for SD-OCT images using region-based C-V model via local similarity factor. Biomed Opt Express 7(2):581–600CrossRef
9.
go back to reference Chiu SJ et al (2015) Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema. Biomed Opt Express 6(4):1172–1194CrossRef Chiu SJ et al (2015) Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema. Biomed Opt Express 6(4):1172–1194CrossRef
10.
go back to reference Srinivasan PP, Heflin SJ, Izatt JA, Arshavsky VY, Farsiu S (2014) Automatic segmentation of up to ten layer boundaries in SD-OCT images of the mouse retina with and without missing layers due to pathology. Biomed Opt Express 5(2):348–365CrossRef Srinivasan PP, Heflin SJ, Izatt JA, Arshavsky VY, Farsiu S (2014) Automatic segmentation of up to ten layer boundaries in SD-OCT images of the mouse retina with and without missing layers due to pathology. Biomed Opt Express 5(2):348–365CrossRef
11.
go back to reference Karri SP, Chakraborthi D, Chatterjee J (2016) Learning layer-specific edges for segmenting retinal layers with large deformations. Biomed Opt Express 7(7):2888–2901CrossRef Karri SP, Chakraborthi D, Chatterjee J (2016) Learning layer-specific edges for segmenting retinal layers with large deformations. Biomed Opt Express 7(7):2888–2901CrossRef
12.
go back to reference Fang L et al (2017) Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search. Biomed Opt Express 8(5):2732–2744CrossRef Fang L et al (2017) Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search. Biomed Opt Express 8(5):2732–2744CrossRef
13.
go back to reference Hamwood J, Alonso-Caneiro D, Read SA, Vincent SJ, Collins MJ (2018) Effect of patch size and network architecture on a convolutional neural network approach for automatic segmentation of OCT retinal layers. Biomed Opt Express 9(7):3049–3066CrossRef Hamwood J, Alonso-Caneiro D, Read SA, Vincent SJ, Collins MJ (2018) Effect of patch size and network architecture on a convolutional neural network approach for automatic segmentation of OCT retinal layers. Biomed Opt Express 9(7):3049–3066CrossRef
14.
go back to reference Kugelman J, Alonso-Caneiro D, Read SA, Vincent SJ, Collins MJ (2018) Automatic segmentation of OCT retinal boundaries using recurrent neural networks and graph search. Biomed Opt Express 9(11):5759–5777CrossRef Kugelman J, Alonso-Caneiro D, Read SA, Vincent SJ, Collins MJ (2018) Automatic segmentation of OCT retinal boundaries using recurrent neural networks and graph search. Biomed Opt Express 9(11):5759–5777CrossRef
15.
go back to reference Roy AG et al (2017) ReLayNet: retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks. Biomed Opt Express 8(8):3627–3642CrossRef Roy AG et al (2017) ReLayNet: retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks. Biomed Opt Express 8(8):3627–3642CrossRef
16.
go back to reference Venhuizen FG et al (2017) Robust total retina thickness segmentation in optical coherence tomography images using convolutional neural networks. Biomed Opt Expres 8(7):3292–3316CrossRef Venhuizen FG et al (2017) Robust total retina thickness segmentation in optical coherence tomography images using convolutional neural networks. Biomed Opt Expres 8(7):3292–3316CrossRef
17.
go back to reference Alonso-Caneiro D et al (2018) Automatic retinal and choroidal boundary segmentation in OCT images using patch-based supervised machine learning methods. In: Asian conference on computer vision (ACCV). Springer, Cham, pp 215–228 Alonso-Caneiro D et al (2018) Automatic retinal and choroidal boundary segmentation in OCT images using patch-based supervised machine learning methods. In: Asian conference on computer vision (ACCV). Springer, Cham, pp 215–228
18.
go back to reference Kugelman J et al (2019) Automatic choroidal segmentation in OCT images using supervised deep learning methods. Sci Rep 9:1–13CrossRef Kugelman J et al (2019) Automatic choroidal segmentation in OCT images using supervised deep learning methods. Sci Rep 9:1–13CrossRef
19.
go back to reference Krizhevsky A, Sutskever I, Hinton G (2012) Imagenet classification with deep convolutional neural networks. In: Annual conference on neural information processing systems (NIPS), pp 1106–1114 Krizhevsky A, Sutskever I, Hinton G (2012) Imagenet classification with deep convolutional neural networks. In: Annual conference on neural information processing systems (NIPS), pp 1106–1114
20.
go back to reference Szegedy C et al. (2015) Going deeper with convolutions. In: Conference on computer vision and pattern recognition (CVPR) Szegedy C et al. (2015) Going deeper with convolutions. In: Conference on computer vision and pattern recognition (CVPR)
22.
23.
go back to reference Shin H et al (2018) Medical image synthesis for data augmentation and anonymization using generative adversarial networks. arXiv:1807.10225 Shin H et al (2018) Medical image synthesis for data augmentation and anonymization using generative adversarial networks. arXiv:​1807.​10225
24.
go back to reference Bailo O, Ham D, Shin YM (2019) Red blood cell image generation for data augmentation using conditional generative adversarial networks. arXiv:1901.06219 Bailo O, Ham D, Shin YM (2019) Red blood cell image generation for data augmentation using conditional generative adversarial networks. arXiv:​1901.​06219
25.
go back to reference Frid-Adar M et al (2018) GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. arXiv:1803.01229 Frid-Adar M et al (2018) GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. arXiv:​1803.​01229
26.
go back to reference Goodfellow IJ et al (2014) Generative adversarial networks. arXiv: 1406.2661 Goodfellow IJ et al (2014) Generative adversarial networks. arXiv: 1406.2661
28.
go back to reference Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434 Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:​1511.​06434
29.
go back to reference Schlegl T, Seeböck P, Waldstein SM, Schmidt-Erfurth U, Langs G (2017) Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: International conference on information processing in medical imaging. Springer, Cham, pp 146–157 Schlegl T, Seeböck P, Waldstein SM, Schmidt-Erfurth U, Langs G (2017) Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: International conference on information processing in medical imaging. Springer, Cham, pp 146–157
30.
go back to reference Halupka KJ et al (2018) Retinal optical coherence tomography image enhancement via deep learning. Biomed Opt Express 9(12):6205–6221CrossRef Halupka KJ et al (2018) Retinal optical coherence tomography image enhancement via deep learning. Biomed Opt Express 9(12):6205–6221CrossRef
31.
go back to reference Huang Y et al (2019) Simultaneous denoising and super-resolution of optical coherence tomography images based on generative adversarial network. Opt Express 27(9):12289–12307CrossRef Huang Y et al (2019) Simultaneous denoising and super-resolution of optical coherence tomography images based on generative adversarial network. Opt Express 27(9):12289–12307CrossRef
32.
go back to reference Romo-Bucheli D et al (2020) Reducing image variability across OCT devices with unsupervised unpaired learning for improved segmentation of retina. Biomed Opt Express 11(1):346–363CrossRef Romo-Bucheli D et al (2020) Reducing image variability across OCT devices with unsupervised unpaired learning for improved segmentation of retina. Biomed Opt Express 11(1):346–363CrossRef
36.
go back to reference Kugelman J et al. (2019) Constructing synthetic chorio-retinal patches using generative adversarial networks. In: 2019 digital image computing: techniques and applications (DICTA). IEEE, pp 1–8 Kugelman J et al. (2019) Constructing synthetic chorio-retinal patches using generative adversarial networks. In: 2019 digital image computing: techniques and applications (DICTA). IEEE, pp 1–8
37.
go back to reference Read SA, Collins MJ, Vincent SJ, Alonso-Caneiro D (2013) Choroidal thickness in childhood. Invest Ophthamol Vis Sci 54(5):3586–3593CrossRef Read SA, Collins MJ, Vincent SJ, Alonso-Caneiro D (2013) Choroidal thickness in childhood. Invest Ophthamol Vis Sci 54(5):3586–3593CrossRef
38.
go back to reference Read SA, Collins MJ, Vincent SJ, Alonso-Caneiro D (2015) Macular retinal layer thickness in childhood. Retina 35:1223–1233CrossRef Read SA, Collins MJ, Vincent SJ, Alonso-Caneiro D (2015) Macular retinal layer thickness in childhood. Retina 35:1223–1233CrossRef
46.
go back to reference Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S (2017) GANS trained by a two time-scale update rule converge to a local nash equilibrium. arXiv:1706.08500 Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S (2017) GANS trained by a two time-scale update rule converge to a local nash equilibrium. arXiv:​1706.​08500
47.
go back to reference Zhao H, Li H, Maurer-Stroh S, Cheng L (2018) Synthetisizing retinal and neuronal images with generative adversarial nets. Med Image Anal 49:14–26CrossRef Zhao H, Li H, Maurer-Stroh S, Cheng L (2018) Synthetisizing retinal and neuronal images with generative adversarial nets. Med Image Anal 49:14–26CrossRef
52.
go back to reference Karras T, Aila T, Laine S, Lehtinen J (2017) Progressive growing of GANs for improved quality, stability, and variation. arXiv:1710.10196 Karras T, Aila T, Laine S, Lehtinen J (2017) Progressive growing of GANs for improved quality, stability, and variation. arXiv:​1710.​10196
Metadata
Title
Data augmentation for patch-based OCT chorio-retinal segmentation using generative adversarial networks
Authors
Jason Kugelman
David Alonso-Caneiro
Scott A. Read
Stephen J. Vincent
Fred K. Chen
Michael J. Collins
Publication date
10-03-2021
Publisher
Springer London
Published in
Neural Computing and Applications / Issue 13/2021
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-021-05826-w

Other articles of this Issue 13/2021

Neural Computing and Applications 13/2021 Go to the issue

Premium Partner