Skip to main content

Advertisement

SpringerLink
Log in

Automatic Classification of Volumetric Optical Coherence Tomography Images via Recurrent Neural Network

  • Original Paper
  • Published:
Sensing and Imaging Aims and scope Submit manuscript

Abstract

Automatic and accurate classification of retinal optical coherence tomography (OCT) images is essential to assist ophthalmologists in the diagnosis and grading of macular diseases. Most existing methods classify 3-D retinal OCT volumes by separately analyzing each single-frame 2-D B-scan, and thus inevitably ignore significant temporal information among B-scans. In this paper, we propose to classify volumetric OCT images via a recurrent neural network (VOCT-RNN) which can fully exploit temporal information among B-scans. Specifically, a deep convolutional neural network is first utilized to automatically extract highly representative features from each individual B-scan of the 3-D retinal OCT images. Then, a long short-term memory network is employed to model the temporal dependencies among B-scans and achieve volumetric OCT classification. The proposed VOCT-RNN can be directly learned from volume-level labels, requiring no detailed annotations at each B-scan. Experimental results on two clinically acquired OCT datasets demonstrate the effectiveness of the proposed VOCT-RNN for volumetric retinal OCT image classification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Pascolini, D., & Mariotti, S. P. (2012). Global estimates of visual impairment: 2010. British Journal of Ophthalmology, 96(5), 614–628.

    Article  Google Scholar 

  2. Neely, D. C., Bray, K. J., Huisingh, C. E., Clark, M. E., McGwin, G. J., & Owsley, C. (2017). Prevalence of undiagnosed age-related macular degeneration in primary eye care. JAMA Ophthalmology, 135(6), 570–575.

    Article  Google Scholar 

  3. Hirai, F. E., Knudtson, M. D., Klein, B. E., & Klein, R. (2008). Clinically significant macular edema and survival in type 1 and type 2 diabetes. American Journal of Ophthalmology, 145(4), 700–706.

    Article  Google Scholar 

  4. Huang, D., Swanson, E. A., Lin, C. P., Schuman, J. S., Stinson, W. G., Chang, W., et al. (1991). Optical coherence tomography. Science, 254(5035), 1178–1181.

    Article  Google Scholar 

  5. Nassif, N., Cense, B., Park, B., Pierce, M., Yun, S., Bouma, B., et al. (2004). In vivo high-resolution video-rate spectral-domain optical coherence tomography of the human retina and optic nerve. Optics Express, 12(3), 367–376.

    Article  Google Scholar 

  6. Drexler, W., & Fujimoto, J. G. (2008). State-of-the-art retinal optical coherence tomography. Progress in Retinal and Eye Research, 27(1), 45–88.

    Article  Google Scholar 

  7. Puliafito, C. A., Hee, M. R., Lin, C. P., Reichel, E., Schuman, J. S., Duker, J. S., et al. (1995). Imaging of macular diseases with optical coherence tomography. Ophthalmology, 102(2), 217–229.

    Article  Google Scholar 

  8. Liu, Y. Y., Chen, M., Ishikawa, H., Wollstein, G., Schuman, J. S., & Rehg, J. M. (2011). Automated macular pathology diagnosis in retinal OCT images using multi-scale spatial pyramid and local binary patterns in texture and shape encoding. Medical Image Analysis, 15(5), 748–759.

    Article  Google Scholar 

  9. Srinivasan, P. P., Kim, L. A., Mettu, P. S., Cousins, S. W., Comer, G. M., Izatt, J. A., et al. (2014). Fully automated detection of diabetic macular edema and dry age-related macular degeneration from optical coherence tomography images. Biomedical Optics Express, 5(10), 3568–3577.

    Article  Google Scholar 

  10. Venhuizen, F. G., van Ginneken, B., van Asten, F., van Grinsven, M. J., Fauser, S., Hoyng, C. B., et al. (2017). Automated staging of age-related macular degeneration using optical coherence tomography. Investigative Ophthalmology & Visual Science, 58(4), 2318–2328.

    Article  Google Scholar 

  11. Lemaitre, G., Rastgoo, M., Massich, J., Cheung, C. Y., Wong, T. Y., Lamoureux, E., et al. (2016). Classification of SD-OCT volumes using local binary patterns: Experimental validation for DME detection. Journal of Ophthalmology, 10(12), 329–346.

    Google Scholar 

  12. Sun, Y., Li, S., & Sun, Z. (2017). Fully automated macular pathology detection in retina optical coherence tomography images using sparse coding and dictionary learning. Journal of Biomedical Optics, 22(1), 12–21.

    MathSciNet  Google Scholar 

  13. Fang, L., Wang, C., Li, S., Yan, J., Chen, X., & Rabbani, H. (2017). Automatic classification of retinal three-dimensional optical coherence tomography images using principal component analysis network with composite kernels. Journal of Biomedical Optics, 22(11), 11–16.

    Article  Google Scholar 

  14. Wang, Y., Zhang, Y., Yao, Z., Zhao, R., & Zhou, F. (2016). Machine learning based detection of age-related macular degeneration (AMD) and diabetic macular edema (DME) from optical coherence tomography (OCT) images. Biomedical Optics Express, 7(12), 4928–4940.

    Article  Google Scholar 

  15. Kafieh, R., Rabbani, H., & Selesnick, I. (2015). Three dimensional data-driven multi scale atomic representation of optical coherence tomography. IEEE Transactions on Medical Imaging, 34(5), 1042–1062.

    Article  Google Scholar 

  16. Fang, L., Li, S., Cunefare, D., & Farsiu, S. (2017). Segmentation based sparse reconstruction of optical coherence tomography images. IEEE Transactions on Medical Imaging, 36(2), 407–421.

    Article  Google Scholar 

  17. Amini, Z., & Rabbani, H. (2017). Optical coherence tomography image denoising using gaussianization transform. Journal of Biomedical Optics, 22(8), 86–101.

    Google Scholar 

  18. Kafieh, R., Rabbani, H., Abramoff, M. D., & Sonka, M. (2013). Curvature correction of retinal OCTs using graph-based geometry detection. Physics in Medicine & Biology, 58(9), 2925–2938.

    Article  Google Scholar 

  19. Zhang, Q., Li, J., Zhuo, L., Zhang, H., & Li, X. (2017). Vehicle color recognition with vehicle-color saliency detection and dual-orientational dimensionality reduction of CNN deep features. Sensing and Imaging, 18(1), 20–29.

    Article  Google Scholar 

  20. Shen, D., Wu, G., & Suk, H. I. (2017). Deep learning in medical image analysis. Annual Review of Biomedical Engineering, 19(1), 221–248.

    Article  Google Scholar 

  21. Qi, D., Hao, C., Lequan, Y., Lei, Z., Jing, Q., Defeng, W., et al. (2016). Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks. IEEE Transactions on Medical Imaging, 35(5), 1182–1195.

    Article  Google Scholar 

  22. Pereira, S., Pinto, A., Alves, V., & Silva, C. A. (2016). Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Transactions on Medical Imaging, 35(5), 1240–1251.

    Article  Google Scholar 

  23. Grinsven, M. J. J. P. V., Ginneken, B. V., Hoyng, C. B., Theelen, T., & Sánchez, C. I. (2016). Fast convolutional neural network training using selective data sampling: Application to hemorrhage detection in color fundus images. IEEE Transactions on Medical Imaging, 35(5), 1273–1284.

    Article  Google Scholar 

  24. Benaddy, M., El Meslouhi, O., Es-saady, Y., & Kardouchi, M. (2019). Handwritten tifinagh characters recognition using deep convolutional neural networks. Sensing and Imaging, 20(1), 9–18.

    Article  Google Scholar 

  25. Fang, L., Cunefare, D., Wang, C., Guymer, R. H., Li, S., & Farsiu, S. (2017). Automatic segmentation of nine retinal layer boundaries in OCT images of non-exudative AMD patients using deep learning and graph search. Biomedical Optics Express, 8(5), 2732–2744.

    Article  Google Scholar 

  26. Lee, C. S., Baughman, D. M., & Lee, A. Y. (2017). Deep learning is effective for classifying normal versus age-related macular degeneration OCT images. Ophthalmology Retina, 1(4), 322–327.

    Article  Google Scholar 

  27. Rasti, R., Mehridehnavi, A., Rabbani, H., & Hajizadeh, F. (2018). Automatic diagnosis of abnormal macula in retinal optical coherence tomography images using wavelet-based convolutional neural network features and random forests classifier. Journal of Biomedical Optics, 23(3), 35–45.

    Article  Google Scholar 

  28. De Fauw, J., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S., et al. (2018). Clinically applicable deep learning for diagnosis and referral in retinal disease. Nature Medicine, 24(9), 1342–1350.

    Article  Google Scholar 

  29. Fang, L., Wang, C., Li, S., Rabbani, H., Chen, X., & Liu, Z. (2019). Attention to lesion: Lesion-aware convolutional neural network for retinal optical coherence tomography image classification. IEEE Transactions on Medical Imaging, 38(8), 1959–1970.

    Article  Google Scholar 

  30. Rasti, R., Rabbani, H., Mehridehnavi, A., & Hajizadeh, F. (2017). Macular OCT classification using a multi-scale convolutional neural network ensemble. IEEE Transactions on Medical Imaging, 37(4), 1024–1034.

    Article  Google Scholar 

  31. Rong, Y., Xiang, D., Zhu, W., Yu, K., Shi, F., Fan, Z., et al. (2019). Surrogate-assisted retinal OCT image classification based on convolutional neural networks. IEEE Journal of Biomedical and Health Informatics, 23(1), 253–263.

    Article  Google Scholar 

  32. Perdomo, O., Rios, H., Rodríguez, F. J., Otálora, S., Meriaudeau, F., Müller, H., et al. (2019). Classification of diabetes-related retinal diseases using a deep learning approach in optical coherence tomography. Computer Methods and Programs in Biomedicine, 178(6), 181–189.

    Article  Google Scholar 

  33. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. In International conference on medical image computing and computer-assisted intervention (pp. 234–241).

  34. Wold, S., Esbensen, K., & Geladi, P. (1987). Principal component analysis. Chemometrics and Intelligent Laboratory Systems, 2(1–3), 37–52.

    Article  Google Scholar 

  35. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of CVPR (pp. 770–778).

  36. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.

    Article  Google Scholar 

  37. Zou, M., & Zhong, Y. (2018). Transfer learning for classification of optical satellite image. Sensing and Imaging, 19(1), 6–14.

    Article  Google Scholar 

  38. Ciompi, F., de Hoop, B., van Riel, S. J., Chung, K., Scholten, E. T., Oudkerk, M., et al. (2015). Automatic classification of pulmonary peri-fissural nodules in computed tomography using an ensemble of 2D views and a convolutional neural network out-of-the-box. Medical Image Analysis, 26(1), 195–202.

    Article  Google Scholar 

  39. Samala, R. K., Chan, H., Hadjiiski, L., Helvie, M. A., Richter, C. D., & Cha, K. H. (2018). Breast cancer diagnosis in digital breast tomosynthesis: effects of training sample size on multi-stage transfer learning using deep neural nets. IEEE Transactions on Medical Imaging, 2(1), 101–113.

    Google Scholar 

  40. Shin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., et al. (2016). Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Transactions on Medical Imaging, 35(5), 1285–1298.

    Article  Google Scholar 

  41. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Proceedings of the conference on advances in neural information processing systems (pp. 1097–1105).

  42. Glorot, X., & Bengio, Y. (2011). Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the international conference on artificial intelligence and statistics (pp. 249–256).

Download references

Acknowledgements

The author would like to thank Prof. Hossein Rabbani from the Department of Biomedical Engineering, Isfahan University of Medical Sciences, for a helpful sharing of OCT dataset.

Author information

Authors and Affiliations

  1. College of Electrical and Information Engineering, Hunan University, Changsha, 410082, China

    Chong Wang & Yuxuan Jin

  2. Department of Ophthalmology, The First Hospital of Hunan University of Chinese Medicine, Changsha, 410082, China

    Xiangdong Chen & Zhimin Liu

Authors
  1. Chong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuxuan Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiangdong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhimin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chong Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, C., Jin, Y., Chen, X. et al. Automatic Classification of Volumetric Optical Coherence Tomography Images via Recurrent Neural Network. Sens Imaging 21, 32 (2020). https://doi.org/10.1007/s11220-020-00299-y

Download citation

  • Received:

  • Revised:

  • Published:

  • DOI: https://doi.org/10.1007/s11220-020-00299-y

Keywords

Search

Navigation