nach oben

Erschienen in:

2019 | OriginalPaper | Buchkapitel

20. Deep Lesion Graph in the Wild: Relationship Learning and Organization of Significant Radiology Image Findings in a Diverse Large-Scale Lesion Database

verfasst von : Ke Yan, Xiaosong Wang, Le Lu, Ling Zhang, Adam P. Harrison, Mohammadhadi Bagheri, Ronald M. Summers

Erschienen in: Deep Learning and Convolutional Neural Networks for Medical Imaging and Clinical Informatics

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Radiologists in their daily work routinely find and annotate significant abnormalities on a large number of radiology images. Such abnormalities, or lesions, have collected over years and stored in hospitals’ picture archiving and communication systems. However, they are basically unsorted and lack semantic annotations like type and location. In this paper, we aim to organize and explore them by learning a deep feature representation for each lesion. A large-scale and comprehensive dataset, DeepLesion, is introduced for this task. DeepLesion contains bounding boxes and size measurements of over 32K lesions. To model their similarity relationship, we leverage multiple supervision information including types, self-supervised location coordinates, and sizes. They require little manual annotation effort but describe useful attributes of the lesions. Then, a triplet network is utilized to learn lesion embeddings with a sequential sampling strategy to depict their hierarchical similarity structure. Experiments show promising qualitative and quantitative results on lesion retrieval, clustering, and classification. The learned embeddings can be further employed to build a lesion graph for various clinically useful applications. An algorithm for intra-patient lesion matching is proposed and validated with experiments.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Automatic Classification and Reporting of Multiple Common Thorax Diseases Using Chest Radiographs

Nächstes Kapitel Simultaneous Super-Resolution and Cross-Modality Synthesis in Magnetic Resonance Imaging

Available at https://nihcc.box.com/v/DeepLesion.

https://nihcc.box.com/v/DeepLesion.

Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434

Bellet A, Habrard A, Sebban M (2013) A survey on metric learning for feature vectors and structured data. arXiv:1306.6709

Bromley J, Guyon I, LeCun Y, Säckinger E, Shah R (1994) Signature verification using a “Siamese” time delay neural network. In: Advances in neural information processing systems, pp 737–744

Cai J, Tang Y, Lu L, Harrison AP, Yan K, Xiao J, Yang L, Summers RM (2018) Accurate weakly-supervised deep lesion segmentation using large-scale clinical annotations: slice-propagated 3D mask generation from 2D RECIST. In: MICCAI. http://arxiv.org/abs/1801.08614CrossRef

Chen W, Chen X, Zhang J, Huang K (2017) Beyond triplet loss: a deep quadruplet network for person re-identification. In: CVPR. https://arxiv.org/abs/1704.01719

Chen X, Gupta A (2015) Webly supervised learning of convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp 1431–1439

Cheng JZ, Ni D, Chou YH, Qin J, Tiu CM, Chang YC, Huang CS, Shen D, Chen CM (2016) Computer-aided diagnosis with deep learning architecture: applications to breast lesions in US images and pulmonary nodules in CT scans. Sci Rep 6(1):24454. https://doi.org/10.1038/srep24454, http://www.nature.com/articles/srep24454

Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, Moore S, Phillips S, Maffitt D, Pringle M, Tarbox L, Prior F (2013) The cancer imaging archive (TCIA): maintaining and operating a public information repository. J Digit Imaging 26(6):1045–1057. https://doi.org/10.1007/s10278-013-9622-7CrossRef

Cornegruta S, Bakewell R, Withey S, Montana G (2016) Modelling radiological language with bidirectional long short-term memory networks. arXiv:1609.08409

10.

Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) ImageNet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, pp 248–255. https://doi.org/10.1109/CVPR.2009.5206848

11.

Eisenhauer E, Therasse P, Bogaerts J, Schwartz LH, Sargent D, Ford R, Dancey J, Arbuck S, Gwyther S, Mooney M, Others (2009) New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). Eur J Cancer 45(2):228–247

12.

Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639):115–118CrossRef

13.

Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (VOC) challenge. Int J Comput Vis 88(2):303–338CrossRef

14.

Gidaris S, Komodakis N (2015) Object detection via a multi-region and semantic segmentation-aware CNN model. In: Proceedings of the IEEE international conference on computer vision, pp 1134–1142

15.

Girshick R (2015) Fast R-CNN. In: Proceedings of the IEEE international conference on computer vision, pp 1440–1448

16.

Greenspan H, van Ginneken B, Summers RM (2016) Guest Editorial deep learning in medical imaging: overview and future promise of an exciting new technique. IEEE Trans Med Imaging 35(5):1153–1159. https://doi.org/10.1109/TMI.2016.2553401, http://ieeexplore.ieee.org/document/7463094/CrossRef

17.

Hofmanninger J, Krenn M, Holzer M, Schlegl T, Prosch H, Langs G (2016) Unsupervised identification of clinically relevant clusters in routine imaging data. In: International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 192–200CrossRef

18.

Hong H, Lee J, Yim Y (2008) Automatic lung nodule matching on sequential CT images. Comput Biol Med 38(5):623–634. https://doi.org/10.1016/j.compbiomed.2008.02.010, http://www.sciencedirect.com/science/article/pii/S0010482508000449CrossRef

19.

Hu P, Ramanan D (2017) Finding tiny faces. In: CVPR. https://doi.org/10.1109/CVPR.2017.166, http://arxiv.org/abs/1612.04402

20.

Krause J, Sapp B, Howard A, Zhou H, Toshev A, Duerig T, Philbin J, Fei-Fei L (2016) The unreasonable effectiveness of noisy data for fine-grained recognition. In: European conference on computer vision. Springer, Berlin, pp 301–320CrossRef

21.

Lee DH (2013) Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on challenges in representation learning, ICML, vol 3, p 2

22.

Li Z, Zhang X, Müller H, Zhang S (2018) Large-scale retrieval for medical image analytics: a comprehensive review. Med Image Anal 43:66–84. https://doi.org/10.1016/j.media.2017.09.007, http://www.sciencedirect.com/science/article/pii/S136184151730138XCrossRef

23.

Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft COCO: common objects in context. In: European conference on computer vision. Springer, Berlin, pp 740–755CrossRef

24.

Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JA, van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88. https://doi.org/10.1016/j.media.2017.07.005, http://www.sciencedirect.com/science/article/pii/S1361841517301135CrossRef

25.

van der Maaten L (2014) Accelerating t-SNE using tree-based algorithms. J Mach Learn Res 15:3221–3245. http://jmlr.org/papers/v15/vandermaaten14a.html

26.

Manning CD, Raghavan P, Schutze H (2008) Introduction to information retrieval. Cambridge University Press, Cambridge. https://nlp.stanford.edu/IR-book/

27.

Moltz JH, D’Anastasi M, Kießling A, Dos Santos DP, Schülke C, Peitgen HO (2012) Workflow-centred evaluation of an automatic lesion tracking software for chemotherapy monitoring by CT. Eur Radiol 22(12):2759–2767CrossRef

28.

Moltz JH, Schwier M, Peitgen HO (2009) A general framework for automatic detection of matching lesions in follow-up CT. In: IEEE international symposium on biomedical imaging: from nano to macro, 2009, ISBI’09. IEEE, pp 843–846

29.

Oh Song H, Xiang Y, Jegelka S, Savarese S (2016) Deep metric learning via lifted structured feature embedding. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4004–4012

30.

Ramos J, Kockelkorn TTJP, Ramos I, Ramos R, Grutters J, Viergever MA, van Ginneken B, Campilho A (2016) Content-based image retrieval by metric learning from radiology reports: application to interstitial lung diseases. IEEE J Biomed Health Inform 20(1):281–292. https://doi.org/10.1109/JBHI.2014.2375491, http://www.ncbi.nlm.nih.gov/pubmed/25438332, http://ieeexplore.ieee.org/document/6966720/CrossRef

31.

Schroff F, Kalenichenko D, Philbin J (2015) FaceNet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 815–823

32.

Setio AAA, Traverso A, De Bel T, Berens MS, van den Bogaard C, Cerello P, Chen H, Dou Q, Fantacci ME, Geurts B et al (2017) Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge. Med Image Anal 42:1–13CrossRef

33.

Sevenster M, Travis AR, Ganesh RK, Liu P, Kose U, Peters J, Chang PJ (2015) Improved efficiency in clinical workflow of reporting measured oncology lesions via pacs-integrated lesion tracking tool. Am J Roentgenol 204(3):576–583CrossRef

34.

Shin HC, Lu L, Kim L, Seff A, Yao J, Summers R (2016) Interleaved text/image deep mining on a large-scale radiology database for automated image interpretation. J Mach Learn Res 17(1–31):2MathSciNet

35.

Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35(5):1285–1298. https://doi.org/10.1109/TMI.2016.2528162, http://www.ncbi.nlm.nih.gov/pubmed/26886976, http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC4890616, http://ieeexplore.ieee.org/document/7404017/CrossRef

36.

Silva JS, Cancela J, Teixeira L (2011) Fast volumetric registration method for tumor follow-up in pulmonary CT exams. J Appl Clin Med Phys 12(2):362–375CrossRef

37.

Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: ICLR 2015

38.

Sohn K (2016) Improved deep metric learning with multi-class N-pair loss objective. In: Neural information processing systems, pp 1–9

39.

Son J, Baek M, Cho M, Han B (2017) Multi-object tracking with quadruplet convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5620–5629

40.

Song HO, Jegelka S, Rathod V, Murphy K (2017) Deep metric learning via facility location. In: IEEE CVPR

41.

Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J (2016) Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans Med Imaging 35(5):1299–1312. https://doi.org/10.1109/TMI.2016.2535302, http://ieeexplore.ieee.org/document/7426826/CrossRef

42.

Tang Y, Harrison AP, Bagheri M, Xiao J, Summers RM (2018) Semi-automatic RECIST labeling on CT scans with cascaded convolutional neural networks. In: MICCAI. http://arxiv.org/abs/1806.09507

43.

Teramoto A, Fujita H, Yamamuro O, Tamaki T (2016) Automated detection of pulmonary nodules in PET/CT images: ensemble false-positive reduction using a convolutional neural network technique. Med Phys 43(6):2821–2827CrossRef

44.

Tsochatzidis L, Zagoris K, Arikidis N, Karahaliou A, Costaridou L, Pratikakis I (2017) Computer-aided diagnosis of mammographic masses based on a supervised content-based image retrieval approach. Pattern Recognit

45.

Vivanti R (2015) Automatic liver tumor segmentation in follow-up ct studies using convolutional neural networks. In: Proceedings of the patch-based methods in medical image processing workshop

46.

Wang X, Lu L, Shin HC, Kim L, Bagheri M, Nogues I, Yao J, Summers RM (2017) Unsupervised joint mining of deep features and image labels for large-scale radiology image categorization and scene recognition. In: 2017 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 998–1007

47.

Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM (2017) ChestX-ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: CVPR. https://doi.org/10.1109/CVPR.2017.369, http://arxiv.org/abs/1705.02315

48.

Wang Z, Yin Y, Shi J, Fang W, Li H, Wang X (2017) Zoom-in-Net: deep mining lesions for diabetic retinopathy detection. Springer International Publishing, Berlin, pp 267–275CrossRef

49.

Wei G, Ma H, Qian W, Qiu M (2016) Similarity measurement of lung masses for medical image retrieval using kernel based semisupervised distance metric. Med Phys 43(12):6259–6269. https://doi.org/10.1118/1.4966030, http://www.ncbi.nlm.nih.gov/pubmed/27908158CrossRef

50.

Yan K, Bagheri M, Summers RM (2018) 3D context enhanced region-based convolutional neural network for end-to-end lesion detection. In: MICCAI, pp 511–519

51.

Yan K, Wang X, Lu L, Summers RM (2018) DeepLesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning. J Med Imaging 5. https://doi.org/10.1117/1.JMI.5.3.036501, https://www.spiedigitallibrary.org/journals/journal-of-medical-imaging/volume-5/issue-03/036501/DeepLesion-automated-mining-of-large-scale-lesion-annotations-and/10.1117/1.JMI.5.3.036501.fullCrossRef

52.

Yan K, Wang X, Lu L, Zhang L, Harrison A, Bagheri M, Summers R (2018) Deep lesion graphs in the wild: relationship learning and organization of significant radiology image findings in a diverse large-scale lesion database. In: CVPR. http://arxiv.org/abs/1711.10535

53.

Zhang H, Shang X, Yang W, Xu H, Luan H, Chua TS (2016) Online collaborative learning for open-vocabulary visual classifiers. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2809–2817

54.

Zhang X, Zhou F, Lin Y, Zhang S (2016) Embedding label structures for fine-grained feature representation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1114–1123

55.

Zhang Z, Xie Y, Xing F, McGough M, Yang L (2017) MDNet: a semantically and visually interpretable medical image diagnosis network. In: CVPR. https://doi.org/10.1109/CVPR.2017.378, http://arxiv.org/abs/1707.02485

56.

Zhu X, Ghahramani Z (2002) Learning from labeled and unlabeled data with label propagation. Technical report CMU-CALD-02-107, Carnegie Mellon University

Titel: Deep Lesion Graph in the Wild: Relationship Learning and Organization of Significant Radiology Image Findings in a Diverse Large-Scale Lesion Database
verfasst von: Ke Yan
Xiaosong Wang
Le Lu
Ling Zhang
Adam P. Harrison
Mohammadhadi Bagheri
Ronald M. Summers
Verlag: Springer International Publishing
Buch: Deep Learning and Convolutional Neural Networks for Medical Imaging and Clinical Informatics
Print ISBN: 978-3-030-13968-1

Electronic ISBN: 978-3-030-13969-8

Copyright-Jahr: 2019
DOI: https://doi.org/10.1007/978-3-030-13969-8_20

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner