Skip to main content

2023 | OriginalPaper | Buchkapitel

A Self-supervised 3D/2D Registration Method for Incomplete DSA Vessels

verfasst von : Yizhou Xu, Cai Meng, Yanggang Li, Ning Li, Longfei Ren, Kun Xia

Erschienen in: Biomedical and Computational Biology

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

For vascular interventional surgery, the preoperative 3D computed tomography (CT) has complete information of vessels but is not convenient for obvervation, while the intraoperative 2D digital subtraction angiography (DSA) is easy for doctors to monitor the vascular conditions real-timely but information is incomplete in each frame. As a result, 2D/3D registration, which is the technology to fuse information from images of different modal, is useful for the guidance of vascular interventional surgery. In this paper, we proposed a self-supervised 2D/3D vascular registration method to improve the performance on DSAs with incomplete vessels. The proposed method contains a rigid and an elastic registration stage, for regressing the 6-dim parameters to obtain a center image and fine-tuning respectively. In addition, a patch-based content loss is introduced to the rigid registration step to give an appropriate similarity measure for images with incomplete vessels, and a masked elastic module is introduced to simulate the incompletion and deformation caused by breath or heart beats on the real vessels in elastic registration. We evaluated our method on both simulated and real images. Experiments prove that our proposed method is effective to register CT and DSA images.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Markelj, P., Tomaževič, D., Likar, B., Pernuš, F.: A review of 3D/2D registration methods for image-guided interventions. Med. Image Anal. 16(3), 642–661 (2012)CrossRefPubMed Markelj, P., Tomaževič, D., Likar, B., Pernuš, F.: A review of 3D/2D registration methods for image-guided interventions. Med. Image Anal. 16(3), 642–661 (2012)CrossRefPubMed
2.
Zurück zum Zitat Hipwell, J.H., et al.: Intensity-based 2-D-3-D registration of cerebral angiograms (2003) Hipwell, J.H., et al.: Intensity-based 2-D-3-D registration of cerebral angiograms (2003)
3.
Zurück zum Zitat Demirci, S., Kutter, O., Manstad-Hulaas, F., Bauernschmitt, R., Navab, N.: Advanced 2D–3D registration for endovascular aortic interventions: addressing dissimilarity in images. In: Medical Imaging 2008: Visualization, Image-Guided Procedures, and Modeling, vol. 6918, p. 69182S. International Society for Optics and Photonics (2008) Demirci, S., Kutter, O., Manstad-Hulaas, F., Bauernschmitt, R., Navab, N.: Advanced 2D–3D registration for endovascular aortic interventions: addressing dissimilarity in images. In: Medical Imaging 2008: Visualization, Image-Guided Procedures, and Modeling, vol. 6918, p. 69182S. International Society for Optics and Photonics (2008)
4.
Zurück zum Zitat Demirci, S., Baust, M., Kutter, O., Manstad-Hulaas, F., Eckstein, H.H., Navab, N.: Disocclusion-based 2D–3D registration for aortic interventions. Comput. Biol. Med. 43(4), 312–322 (2013)CrossRefPubMed Demirci, S., Baust, M., Kutter, O., Manstad-Hulaas, F., Eckstein, H.H., Navab, N.: Disocclusion-based 2D–3D registration for aortic interventions. Comput. Biol. Med. 43(4), 312–322 (2013)CrossRefPubMed
5.
Zurück zum Zitat Miao, S., Liao, R., Zheng, Y.: A hybrid method for 2-D/3-D registration between 3-D volumes and 2-D angiography for trans-catheter aortic valve implantation (TAVI). In: 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 1215–1218. IEEE (2011) Miao, S., Liao, R., Zheng, Y.: A hybrid method for 2-D/3-D registration between 3-D volumes and 2-D angiography for trans-catheter aortic valve implantation (TAVI). In: 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 1215–1218. IEEE (2011)
6.
Zurück zum Zitat Raheem, A., Carrell, T., Modarai, B., Penney, G.: Non-rigid 2D–3D image registration for use in endovascular repair of abdominal aortic aneurysms. In: Medical Image Understanding and Analysis, pp. 153–157 (2010) Raheem, A., Carrell, T., Modarai, B., Penney, G.: Non-rigid 2D–3D image registration for use in endovascular repair of abdominal aortic aneurysms. In: Medical Image Understanding and Analysis, pp. 153–157 (2010)
7.
Zurück zum Zitat Hunsche, S., et al.: Intensity-based 2D 3D registration for lead localization in robot guided deep brain stimulation. Phys. Med. Biol. 62(6), 2417 (2017)CrossRefPubMed Hunsche, S., et al.: Intensity-based 2D 3D registration for lead localization in robot guided deep brain stimulation. Phys. Med. Biol. 62(6), 2417 (2017)CrossRefPubMed
8.
Zurück zum Zitat Meng, C., Wang, Q., Guan, S., Sun, K., Liu, B.: 2D–3D registration with weighted local mutual information in vascular interventions. IEEE Access 7, 162629–162638 (2019)CrossRef Meng, C., Wang, Q., Guan, S., Sun, K., Liu, B.: 2D–3D registration with weighted local mutual information in vascular interventions. IEEE Access 7, 162629–162638 (2019)CrossRef
9.
Zurück zum Zitat Yang, K., et al.: A novel 2D/3D hierarchical registration framework via principal-directional Fourier transform operator. Phys. Med. Biol. 66(6), 065030 (2021)CrossRefPubMed Yang, K., et al.: A novel 2D/3D hierarchical registration framework via principal-directional Fourier transform operator. Phys. Med. Biol. 66(6), 065030 (2021)CrossRefPubMed
10.
Zurück zum Zitat Groher, M., Zikic, D., Navab, N.: Deformable 2D–3D registration of vascular structures in a one view scenario. IEEE Trans. Med. Imaging 28(6), 847–860 (2009)CrossRefPubMed Groher, M., Zikic, D., Navab, N.: Deformable 2D–3D registration of vascular structures in a one view scenario. IEEE Trans. Med. Imaging 28(6), 847–860 (2009)CrossRefPubMed
11.
Zurück zum Zitat Rivest-Henault, D., Sundar, H., Cheriet, M.: Nonrigid 2D/3D registration of coronary artery models with live fluoroscopy for guidance of cardiac interventions. IEEE Trans. Med. Imaging 31(8), 1557–1572 (2012)CrossRefPubMed Rivest-Henault, D., Sundar, H., Cheriet, M.: Nonrigid 2D/3D registration of coronary artery models with live fluoroscopy for guidance of cardiac interventions. IEEE Trans. Med. Imaging 31(8), 1557–1572 (2012)CrossRefPubMed
12.
Zurück zum Zitat Xu, D., et al.: Single-view 2D/3D registation for X-ray guided bronchoscopy. In: 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 233–236. IEEE (2010) Xu, D., et al.: Single-view 2D/3D registation for X-ray guided bronchoscopy. In: 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 233–236. IEEE (2010)
13.
Zurück zum Zitat Ghafurian, S., Hacihaliloglu, I., Metaxas, D.N., Tan, V., Li, K.: 3D/2D image registration using weighted histogram of gradient directions. In: Medical Imaging 2015: Image-Guided Procedures, Robotic Interventions, and Modeling, vol. 9415, p. 94151Z. International Society for Optics and Photonics (2015) Ghafurian, S., Hacihaliloglu, I., Metaxas, D.N., Tan, V., Li, K.: 3D/2D image registration using weighted histogram of gradient directions. In: Medical Imaging 2015: Image-Guided Procedures, Robotic Interventions, and Modeling, vol. 9415, p. 94151Z. International Society for Optics and Photonics (2015)
14.
Zurück zum Zitat Zhu, J., et al.: Heuristic tree searching for pose-independent 3D/2D rigid registration of vessel structures. Phys. Med. Biol. 65(5), 055010 (2020)CrossRefPubMed Zhu, J., et al.: Heuristic tree searching for pose-independent 3D/2D rigid registration of vessel structures. Phys. Med. Biol. 65(5), 055010 (2020)CrossRefPubMed
15.
Zurück zum Zitat Yoon, S., Yoon, C.H., Lee, D.: Topological recovery for non-rigid 2D/3D registration of coronary artery models. Comput. Methods Programs Biomed. 200, 105922 (2021)CrossRefPubMed Yoon, S., Yoon, C.H., Lee, D.: Topological recovery for non-rigid 2D/3D registration of coronary artery models. Comput. Methods Programs Biomed. 200, 105922 (2021)CrossRefPubMed
16.
Zurück zum Zitat Chou, C.R., Frederick, B., Mageras, G., Chang, S., Pizer, S.: 2D/3D image registration using regression learning. Comput. Vis. Image Underst. 117(9), 1095–1106 (2013)CrossRefPubMedPubMedCentral Chou, C.R., Frederick, B., Mageras, G., Chang, S., Pizer, S.: 2D/3D image registration using regression learning. Comput. Vis. Image Underst. 117(9), 1095–1106 (2013)CrossRefPubMedPubMedCentral
17.
Zurück zum Zitat Gouveia, A.R., Metz, C., Freire, L., Almeida, P., Klein, S.: Registration-by-regression of coronary CTA and X-ray angiography. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 5(3), 208–220 (2017)CrossRef Gouveia, A.R., Metz, C., Freire, L., Almeida, P., Klein, S.: Registration-by-regression of coronary CTA and X-ray angiography. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 5(3), 208–220 (2017)CrossRef
18.
Zurück zum Zitat Miao, S., Wang, Z.J., Liao, R.: A CNN regression approach for real-time 2D/3D registration. IEEE Trans. Med. Imaging 35(5), 1352–1363 (2016)CrossRef Miao, S., Wang, Z.J., Liao, R.: A CNN regression approach for real-time 2D/3D registration. IEEE Trans. Med. Imaging 35(5), 1352–1363 (2016)CrossRef
19.
Zurück zum Zitat Miao, S., Wang, Z.J., Zheng, Y., Liao, R.: Real-time 2D/3D registration via CNN regression. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp. 1430–1434. IEEE (2016) Miao, S., Wang, Z.J., Zheng, Y., Liao, R.: Real-time 2D/3D registration via CNN regression. In: 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pp. 1430–1434. IEEE (2016)
20.
Zurück zum Zitat Zheng, J., Miao, S., Wang, Z.J., Liao, R.: Pairwise domain adaptation module for CNN-based 2-D/3-D registration. J. Med. Imaging 5(2), 021204 (2018)CrossRef Zheng, J., Miao, S., Wang, Z.J., Liao, R.: Pairwise domain adaptation module for CNN-based 2-D/3-D registration. J. Med. Imaging 5(2), 021204 (2018)CrossRef
21.
Zurück zum Zitat Guan, S., Meng, C., Xie, Y., Wang, Q., Sun, K., Wang, T.: Deformable cardiovascular image registration via multi-channel convolutional neural network. IEEE Access 7, 17524–17534 (2019)CrossRef Guan, S., Meng, C., Xie, Y., Wang, Q., Sun, K., Wang, T.: Deformable cardiovascular image registration via multi-channel convolutional neural network. IEEE Access 7, 17524–17534 (2019)CrossRef
22.
Zurück zum Zitat Miao, S., et al.: Dilated FCN for multi-agent 2D/3D medical image registration. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018) Miao, S., et al.: Dilated FCN for multi-agent 2D/3D medical image registration. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
23.
Zurück zum Zitat Toth, D.D., et al.: 3D/2D model-to-image registration by imitation learning for cardiac procedures. Int. J. Comput. Assist. Radiol. Surg. 13(8), 1141–1149 (2018)CrossRefPubMedPubMedCentral Toth, D.D., et al.: 3D/2D model-to-image registration by imitation learning for cardiac procedures. Int. J. Comput. Assist. Radiol. Surg. 13(8), 1141–1149 (2018)CrossRefPubMedPubMedCentral
24.
25.
Zurück zum Zitat Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: An unsupervised learning model for deformable medical image registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9252–9260 (2018) Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: An unsupervised learning model for deformable medical image registration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9252–9260 (2018)
27.
Zurück zum Zitat Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K.: Spatial transformer networks. arXiv preprint arXiv:1506.02025 (2015) Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K.: Spatial transformer networks. arXiv preprint arXiv:​1506.​02025 (2015)
28.
Zurück zum Zitat Hu, Y., et al.: Label-driven weakly-supervised learning for multimodal deformable image registration. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 1070–1074. IEEE (2018) Hu, Y., et al.: Label-driven weakly-supervised learning for multimodal deformable image registration. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 1070–1074. IEEE (2018)
29.
Zurück zum Zitat Zeng, Q., et al.: Label-driven magnetic resonance imaging (MRI)-transrectal ultrasound (TRUS) registration using weakly supervised learning for MRI-guided prostate radiotherapy. Phys. Med. Biol. 65(13), 135002 (2020)CrossRefPubMedPubMedCentral Zeng, Q., et al.: Label-driven magnetic resonance imaging (MRI)-transrectal ultrasound (TRUS) registration using weakly supervised learning for MRI-guided prostate radiotherapy. Phys. Med. Biol. 65(13), 135002 (2020)CrossRefPubMedPubMedCentral
30.
Zurück zum Zitat Li, P., Pei, Y., Guo, Y., Ma, G., Xu, T., Zha, H.: Non-rigid 2D–3D registration using convolutional autoencoders. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 700–704. IEEE (2020) Li, P., Pei, Y., Guo, Y., Ma, G., Xu, T., Zha, H.: Non-rigid 2D–3D registration using convolutional autoencoders. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), pp. 700–704. IEEE (2020)
31.
Zurück zum Zitat Zhang, Y.: An unsupervised 2D–3D deformable registration network (2D3D-RegNet) for cone-beam CT estimation. Phys. Med. Biol. 66(7), 074001 (2021)CrossRef Zhang, Y.: An unsupervised 2D–3D deformable registration network (2D3D-RegNet) for cone-beam CT estimation. Phys. Med. Biol. 66(7), 074001 (2021)CrossRef
32.
Zurück zum Zitat He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016) He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
33.
Zurück zum Zitat Levoy, M.: Display of surfaces from volume data. IEEE Comput. Graphics Appl. 8(3), 29–37 (1988)CrossRef Levoy, M.: Display of surfaces from volume data. IEEE Comput. Graphics Appl. 8(3), 29–37 (1988)CrossRef
34.
Zurück zum Zitat Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018) Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:​1810.​04805 (2018)
35.
Zurück zum Zitat He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. arXiv preprint arXiv:2111.06377 (2021) He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. arXiv preprint arXiv:​2111.​06377 (2021)
36.
Zurück zum Zitat Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 249–256. JMLR Workshop and Conference Proceedings (2010) Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 249–256. JMLR Workshop and Conference Proceedings (2010)
37.
Zurück zum Zitat Van de Kraats, E.B., Penney, G.P., Tomazevic, D., Van Walsum, T., Niessen, W.J.: Standardized evaluation methodology for 2-D-3-D registration. IEEE Trans. Med. Imaging 24(9), 1177–1189 (2005)CrossRefPubMed Van de Kraats, E.B., Penney, G.P., Tomazevic, D., Van Walsum, T., Niessen, W.J.: Standardized evaluation methodology for 2-D-3-D registration. IEEE Trans. Med. Imaging 24(9), 1177–1189 (2005)CrossRefPubMed
Metadaten
Titel
A Self-supervised 3D/2D Registration Method for Incomplete DSA Vessels
verfasst von
Yizhou Xu
Cai Meng
Yanggang Li
Ning Li
Longfei Ren
Kun Xia
Copyright-Jahr
2023
DOI
https://doi.org/10.1007/978-3-031-25191-7_2

Premium Partner