Abstract
The detailed analysis of secondary envelopment of the Human betaherpesvirus 5/human cytomegalovirus (HCMV) from transmission electron microscopy (TEM) images is an important step towards understanding the mechanisms underlying the formation of infectious virions. As a step towards a software-based quantification of different stages of HCMV virion morphogenesis in TEM, we developed a transfer learning approach based on convolutional neural networks (CNNs) that automatically detects HCMV nucleocapsids in TEM images. In contrast to existing image analysis techniques that require time-consuming manual definition of structural features, our method automatically learns discriminative features from raw images without the need for extensive pre-processing. For this a constantly growing TEM image database of HCMV infected cells was available which is unique regarding image quality and size in the terms of virological EM. From the two investigated types of transfer learning approaches, namely feature extraction and fine-tuning, the latter enabled us to successfully detect HCMV nucleocapsids in TEM images. Our detection method has outperformed some of the existing image analysis methods based on discriminative textural indicators and radial density profiles for virus detection in TEM images. In summary, we could show that the method of transfer learning can be used for an automated detection of viral capsids in TEM images with high specificity using standard computers. This method is highly adaptable and in future could be easily extended to automatically detect and classify virions of other viruses and even distinguish different virion maturation stages.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdolmanafi A, Duong L, Dahdah N, Cheriet F (2017) Deep feature learning for automatic tissue classification of coronary artery using optical coherence tomography. Biomed Opt Express 8:1203–1220. https://doi.org/10.1364/BOE.8.001203
Adelson EH, Anderson CH, Bergen JR, Burt PJ, Ogden JM (1984) Pyramid methods in image processing. RCA Eng 29(6):33–41
Andrei G, De Clercq E, Snoeck R (2009) Drug targets in cytomegalovirus infection. Infect Disord Drug Targets 9:201–222. https://doi.org/10.2174/187152609787847758
Bengio Y (2012) Practical recommendations for gradient-based training of deep architectures. In: In Montavon G, B.Orr G, Müller KR (eds) Neural networks: tricks of the trade. Lecture notes in computer science, vol 7700. Springer, Berlin, pp 437–478. https://doi.org/10.1007/978-3-642-35289-8_26
Cappadona I, Villinger C, Schutzius G, Mertens T, von Einem J (2015) Human cytomegalovirus pUL47 modulates tegumentation and capsid accumulation at the viral assembly complex. J Virol 89:7314–7328. https://doi.org/10.1128/JVI.00603-15
Chollet FC (2017) Deep learning with python, 1st edn. Manning Publications, New York, pp 160–166
Deng J, Dong W, Socher R, Li LJ, Li K, Li FF (2009) ImageNet: a large-scale hierarchical image database. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), Miami, pp 248–255. https://doi.org/10.1109/CVPR.2009.5206848
Dietz AN, Villinger C, Becker S, Frick M, von Einem J (2018) A tyrosine-based trafficking motif of the tegument protein pUL71 is crucial for human cytomegalovirus secondary envelopment. J Virol 92:e00907–e00917. https://doi.org/10.1128/JVI.00907-17
Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542:115–118. https://doi.org/10.1038/nature21056
Ghafoorian M, Mehrtash A, Kapur T, Karssemeijer N, Marchiori E, Pesteie M, Guttmann CRG, Leeuw FE, Tempany CM, van Ginneken B, Fedorov A, Abolmaesumi P, Platel B, Wells WM III (2017) Transfer learning for domain adaptation in MRI: application in brain lesion segmentation. In: Descoteaux M, Maier-Hein L, Franz A (eds) Proceedings of the 20th medical image computing and computer-assisted intervention (MICCAI) international conference, Part 3, Quebec, pp 516–524. https://doi.org/10.1007/978-3-319-66179-7
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, pp 770–778. https://doi.org/10.1109/CVPR.2016.90
Ito E, Sato T, Sano D, Utagawa E, Kato T (2018) Virus particle detection by convolutional neural network in transmission electron microscopy images. Food Environ Virol 10:201–208. https://doi.org/10.1007/s12560-018-9335-7
Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: Proceedings of the international conference on learning representations, San Diego, pp 1–15. https://arxiv.org/abs/1412.6980v8
Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks, vol 1. In: Proceedings of the 25th international conference on neural information processing systems, Lake Tahoe, pp 1097–1105. https://doi.org/10.1145/3065386
Kylberg G, Uppström M, Hedlund KO, Borgefors G, Sintorn IM (2012) Segmentation of virus particle candidates in transmission electron microscopy images. J Microsc 245:140–147. https://doi.org/10.1111/j.1365-2818.2011.03556.x
Lam C, Yu C, Huang L, Rubin D (2018) Retinal lesion detection with deep learning using image patches. Invest Ophthalmol Vis Sci 59:590–596. https://doi.org/10.1167/iovs.17-22721
LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognitions. Proc IEEE 86:2278–2324. https://doi.org/10.1109/5.726791
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444. https://doi.org/10.1038/nature14539
Li R, Si D, Zeng T, Ji S, He J (2016) Deep convolutional neural networks for detecting secondary structures in protein density maps from cryo-electron microscopy. In: Proceedings of IEEE international conference on bioinformatics and biomedicine, Shenzhen, pp 41–46. https://doi.org/10.1109/BIBM.2016.7822490
Mercorelli B, Lembo D, Palù G, Loregian A (2011) Early inhibitors of human cytomegalovirus: state-of-art and therapeutic perspectives. Pharmacol Ther 131:309–329. https://doi.org/10.1016/J.PHARMTHERA.2011.04.007
Mocarski E, Shenk T, Pass RF (2007) Cytomegoloviruses. In: Knipe DM, Howley PM, Griffin DE, Lamb RA, Martin MA, Roizman B, Straus SE (eds) Field virology, 5th edn. Lippincott Williams & Wilkins, Philadelphia, pp 2701–2771
Modarres MH, Aversa R, Cozzini S, Ciancio R, Leto A, Brandino GP (2017) Neural network for nanoscience scanning electron microscope image recognition. Sci Rep. https://doi.org/10.1038/s41598-017-13565-z
Peddie CJ, Collinson LM (2014) Exploring the third dimension: volume electron microscopy comes of age. Micron 61:9–19. https://doi.org/10.1016/j.micron.2014.01.009
Proença MC, Nunes JFM, de Matos APA (2013) Texture indicators for segmentation of polyomavirus particles in transmission electron microscopy images. Microsc Microanal 19:1170–1182. https://doi.org/10.1017/S1431927613001736
Ribeiro E, Uhl A, Wimmer G, Häfner M (2016) Exploring deep learning and transfer learning for colonic polyp classification. Comput Math Methods Med 2016:1–16. https://doi.org/10.1155/2016/6584725
Ribli D, Horváth A, Unger Z, Pollner P, Csabai I (2018) Detecting and classifying lesions in mammograms with deep learning. Sci Rep 8:4165. https://doi.org/10.1038/s41598-018-22437-z
Roels J, De Vylder J, Aelterman J, Saeys Y, Philips W (2017) Convolutional neural network pruning to accelerate membrane segmentation in electron microscopy. In: Proceedings of the 14th international symposium on IEEE biomedical imaging, Melbourne, pp 633–637. https://doi.org/10.1109/ISBI.2017.7950600
Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis 115:211–252. https://doi.org/10.1007/s11263-015-0816-y
Ryner M, Strömberg JO, Söderberg-Nauclér C, Homman-Loudiyi M (2006) Identification and classification of human cytomegalovirus capsids in textured electron micrographs using deformed template matching. J Virol 3:57. https://doi.org/10.1186/1743-422X-3-57
Schauflinger M, Fischer D, Schreiber A, Chevillotte M, Walther P, Mertens T, von Einem J (2011) The tegument protein UL71 of human cytomegalovirus is involved in late envelopment and affects multivesicular bodies. J Virol 85:3821–3832. https://doi.org/10.1128/JVI.01540-10
Schauflinger M, Villinger C, Mertens T, Walther P, von Einem J (2013) Analysis of human cytomegalovirus secondary envelopment by advanced electron microscopy. Cell Microbiol 15:305–314. https://doi.org/10.1111/cmi.12077
Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35:1285–1298. https://doi.org/10.1109/tmi.2016.2528162
Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. https://arxiv.org/abs/1409.1556
Sintorn IM, Homman-Loudiyi M, Söderberg-Nauclér C, Borgefors G (2004) A refined circular template matching method for classification of human cytomegalovirus capsids in TEM images. Comput Methods Programs Biomed 76:95–102. https://doi.org/10.1016/j.cmpb.2004.03.006
Sinzger C, Digel M, Jahn G (2008) Cytomegalovirus cell tropism. In: Compans RW, Malissen B, Aktories K, Rappuoli R, Galan JE, Ahmed R, Palme K, Casadevall A, Garcia-Sastre A, Iwasaki A, Akira S (eds) Curr Top Microbiol Immunol 325:63–83. https://doi.org/10.1007/978-3-540-77349-8_4
Sousa RG, Esteves T, Rocha S, Figueiredo F, De Sá JM, Alexandre LA, Santos JM, Silva LM (2015) Transfer learning for the recognition of immunogold particles in TEM imaging. In: Rojas I, Joya G, Catala A (eds) Advances in computational intelligence. Lecture notes in computer science, vol 9094. Springer, Cham, pp 374–384. https://doi.org/10.1007/978-3-319-19258-1_32
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the Inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, pp 2818–2826. https://doi.org/10.1109/cvpr.2016.308
Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J (2016) Convolutional neural networks for medical image analysis: full training or Fine tuning? IEEE Trans Med Imaging 35:1299–1312. https://doi.org/10.1109/TMI.2016.2535302
Van Der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res Vol 9:2579–2605
Villinger C, Schauflinger M, Gregorius H, Kranz C, Höhn K, Nafeey S, Walther P (2014) Three-dimensional imaging of adherent cells using FIB/SEM and STEM. Methods Mol Biol 1117:617–638. https://doi.org/10.1007/978-1-62703-776-1_27
Villinger C, Neusser G, Kranz C, Walther P, Mertens T (2015) 3D analysis of HCMV induced-nuclear membrane structures by FIB/SEM tomography: insight into an unprecedented membrane morphology. Viruses 7:5686–5704. https://doi.org/10.3390/v7112900
Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ (eds) Advances in neural information processing systems, vol 27. NIPS 2014, Montreal, pp 3320–3328
Zhu Y, Ouyang Q, Mao Y (2017) A deep convolutional neural network approach to single-particle recognition in cryo-electron microscopy. BMC Bioinform 18:348. https://doi.org/10.1186/s12859-017-1757-y
Acknowledgements
This work was partially funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) Projektnummer 316249678-SFB 1279.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Devan, K.S., Walther, P., von Einem, J. et al. Detection of herpesvirus capsids in transmission electron microscopy images using transfer learning. Histochem Cell Biol 151, 101–114 (2019). https://doi.org/10.1007/s00418-018-1759-5
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00418-018-1759-5