nach oben

International Journal of Computer Assisted Radiology and Surgery

Erschienen in:

14.11.2018 | Original Article

Combining MRF-based deformable registration and deep binary 3D-CNN descriptors for large lung motion estimation in COPD patients

verfasst von: Max Blendowski, Mattias P. Heinrich

Erschienen in: International Journal of Computer Assisted Radiology and Surgery | Ausgabe 1/2019

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Purpose

Deep convolutional neural networks in their various forms are currently achieving or outperforming state-of-the-art results on several medical imaging tasks. We aim to make these developments available to the so far unsolved task of accurate correspondence finding—especially with regard to image registration.

Methods

We propose a two-step hybrid approach to make deep learned features accessible to a discrete optimization-based registration method. In a first step, in order to extract expressive binary local descriptors, we train a deep network architecture on a patch-based landmark retrieval problem as auxiliary task. As second step at runtime within a MRF-regularised dense displacement sampling, their binary nature enables highly efficient similarity computations, thus making them an ideal candidate to replace the so far used handcrafted local feature descriptors during the registration process.

Results

We evaluate our approach on finding correspondences between highly non-rigidly deformed lung CT scans from different breathing states. Although the CNN-based descriptors excell at an auxiliary learning task for finding keypoint correspondences, self-similarity-based descriptors yield more accurate registration results. However, a combination of both approaches turns out to generate the most robust features for registration.

Conclusion

We present a three-dimensional framework for large lung motion estimation based on the combination of CNN-based and handcrafted descriptors efficiently employed in a discrete registration method. Achieving best results by combining learned and handcrafted features encourages further research in this direction.

Vorheriger Artikel Deep learning-based detection of motion artifacts in probe-based confocal laser endomicroscopy images

Nächster Artikel A machine learning pipeline for internal anatomical landmark embedding based on a patient surface model

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Avants BB, Epstein CL, Grossman M, Gee JC (2008) Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med Image Anal 12(1):26–41CrossRefPubMed

Blendowski M, Heinrich MP (2018) 3D-CNNs for deep binary descriptor learning in medical volume data. In: Maier A, Deserno T, Handels H, Maier-Hein K, Palm C, Tolxdorff T (eds) Bildverarbeitung für die Medizin. Springer, Berlin, Heidelberg, pp 23–28. https://doi.org/10.1007/978-3-662-56537-7_19

Brox T, Bregler C, Malik J (2009) Largedisplacement optical flow. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp. 41–48

Calonder M, Lepetit V, Strecha C, Fua P (2010) Brief: binary robust independent elementary features. In: European conference on computer vision. Springer, pp 778–792

Castillo R, Castillo E, Guerra R, Johnson VE, McPhail T, Garg AK, Guerrero T (2009) A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets. Phys Med Biol 54(7):1849CrossRefPubMed

Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O (2016) 3d U-net: learning dense volumetric segmentation from sparse annotation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 424–432

Conjeti S, Roy AG, Katouzian A, Navab N (2017) Hashing with residual networks for image retrieval. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 541–549

de Vos BD, Berendsen FF, Viergever MA, Staring M, Išgum I (2017) End-to-end unsupervised deformable image registration with a convolutional neural network. In: Cardoso M et al (eds) Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, Cham, pp 204–212. https://doi.org/10.1007/978-3-319-67558-9_24

Dosovitskiy A, Fischer P, IlgE, Hausser P, Hazirbas C, Golkov V, van der Smagt P,Cremers D, Brox T (2015) Flownet: learning optical flow with convolutional networks. In: Proceedings of the IEEE international conference on computer vision, pp 2758–2766

10.

Dou Q, Chen H, Yu L, Qin J, Heng PA (2017) Multilevel contextual 3-D CNNS for false positive reduction in pulmonary nodule detection. IEEE Trans Biomed Eng 64(7):1558–1567CrossRefPubMed

11.

Eilertsen G, Forssén PE, Unger J (2017) Briefmatch: dense binary feature matching for real-time optical flow estimation. In: Scandinavian conference on image analysis. Springer, pp 221–233

12.

Felzenszwalb PF, Huttenlocher DP (2005) Pictorial structures for object recognition. Int J Comput Vis 61(1):55–79CrossRef

13.

Glocker B, Komodakis N, Tziritas G, Navab N, Paragios N (2008) Dense image registration through MRFS and efficient linear programming. Med Image Anal 12(6):731–741CrossRefPubMed

14.

He K, Zhang X, Ren S, Sun J (2016) Deepresidual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

15.

Heinrich MP, Blendowski M,Oktay O (2018) Ternarynet: faster deep model inference without GPUS formedical 3D segmentation using sparse and binary convolutions. arXivpreprint https://arxiv.org/abs/801.09449

16.

Heinrich MP, Handels H, Simpson IJ (2015) Estimating large lung motion in COPD patients by symmetric regularised correspondence fields. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 338–345

17.

Heinrich MP, Jenkinson M, Brady M, Schnabel JA (2013) Mrf-based deformable registration and ventilation estimation of lung CT. IEEE Trans Med Imaging 32(7):1239–1248CrossRefPubMed

18.

Heinrich MP, Jenkinson M, Papież BW, Brady M, Schnabel JA (2013) Towards realtime multimodal fusion for image-guided interventions using self-similarities. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 187–194

19.

Heinrich MP, Oktay O (2017) Briefnet: deep pancreas segmentation using binary sparse convolutions. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 329–337

20.

Hu Y, Modat M, Gibson E, Li W, Ghavami N, Bonmati E, Wang G, Bandula S, Moore CM, Emberton M et al (2018) Weakly-supervised convolutional neural networks for multimodal image registration. Med Image Anal 49:1–13CrossRefPubMedPubMedCentral

21.

Huang G, Liu Z, Weinberger KQ, van der Maaten L (2017) Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Patternrecognition 1:3

22.

Liu H, Wang R, Shan S, Chen X (2016) Deep supervised hashing for fast image retrieval. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2064–2072

23.

Milletari F, Navab N, Ahmadi SA (2016) V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth international conference on 3D vision (3DV). IEEE, pp 565–571

24.

Modat M, Ridgway GR, Taylor ZA, Lehmann M, Barnes J, Hawkes DJ, Fox NC, Ourselin S (2010) Fast free-form deformation using graphics processing units. Comput Methods Prog Biomed 98(3):278–284CrossRef

25.

Muenzing SE, van Ginneken B, Viergever MA, Pluim JP (2014) Dirboost-an algorithm for boosting deformable image registration: application to lung CT intra-subject registration. Med Image Anal 18(3):449–459CrossRefPubMed

26.

Muła W, Kurz N, Lemire D (2017) Faster population counts using avx2 instructions. Comput J 61(1):111–120CrossRef

27.

Reinhardt JM, Ding K, Cao K, Christensen GE, Hoffman EA, Bodas SV (2008) Registration-based estimates of local lung tissue expansion compared to xenon CT measures of specific ventilation. Med Image Anal 12(6):752–763CrossRefPubMedPubMedCentral

28.

Rohé MM, Datar M, Heimann T, Sermesant M, Pennec X (2017) SVF-net: learning deformable image registration using shape matching. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 266–274

29.

Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

30.

Rühaak J, Polzin T, Heldmann S, Simpson IJ, Handels H, Modersitzki J, Heinrich MP (2017) Estimation of large motion in lung CT by integrating regularized keypoint correspondences into dense deformable registration. IEEE Trans Med Imaging 36(8):1746–1757CrossRefPubMed

31.

Weinzaepfel P, Revaud J, Harchaoui Z, Schmid C (2013) Deepflow: large displacement optical flow with deep matching. In: 2013 IEEE International conference on computer vision (ICCV). IEEE, pp 1385–1392

32.

Zhang Y, Ozay M, Li S, OkataniT (2017) Truncating wide networks using binary tree architectures. arXivpreprint https://arxiv.org/abs/1704.00509

Titel: Combining MRF-based deformable registration and deep binary 3D-CNN descriptors for large lung motion estimation in COPD patients
verfasst von: Max Blendowski
Mattias P. Heinrich
Publikationsdatum: 14.11.2018
Verlag: Springer International Publishing
Erschienen in: International Journal of Computer Assisted Radiology and Surgery / Ausgabe 1/2019
Print ISSN: 1861-6410
Elektronische ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-018-1888-2

Springer Professional

Abstract

Purpose

Methods

Results

Conclusion

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Weitere Artikel der Ausgabe 1/2019

Automatic bone segmentation in whole-body CT images

Volumetric analysis of intracranial vessels: a novel tool for evaluation of cerebral vasospasm

Deep learning-based detection of motion artifacts in probe-based confocal laser endomicroscopy images

Traditional machine learning for limited angle tomography

The accuracy of image-based safety analysis for robotic cochlear implantation

A machine learning pipeline for internal anatomical landmark embedding based on a patient surface model