Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

16-07-2020 | Original Article

Automatic needle tracking using Mask R-CNN for MRI-guided percutaneous interventions

Authors: Xinzhou Li, Adam S. Young, Steven S. Raman, David S. Lu, Yu-Hsiu Lee, Tsu-Chin Tsao, Holden H. Wu

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 10/2020

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Purpose

Accurate needle tracking provides essential information for MRI-guided percutaneous interventions. Passive needle tracking using MR images is challenged by variations of the needle-induced signal void feature in different situations. This work aimed to develop an automatic needle tracking algorithm for MRI-guided interventions based on the Mask Region Proposal-Based Convolutional Neural Network (R-CNN).

Methods

Mask R-CNN was adapted and trained to segment the needle feature using 250 intra-procedural images from 85 MRI-guided prostate biopsy cases and 180 real-time images from MRI-guided needle insertion in ex vivo tissue. The segmentation masks were passed into the needle feature localization algorithm to extract the needle feature tip location and axis orientation. The proposed algorithm was tested using 208 intra-procedural images from 40 MRI-guided prostate biopsy cases, and 3 real-time MRI datasets in ex vivo tissue. The algorithm results were compared with human-annotated references.

Results

In prostate datasets, the proposed algorithm achieved needle feature tip localization error with median Euclidean distance (dxy) of 0.71 mm and median difference in axis orientation angle (dθ) of 1.28°, respectively. In 3 real-time MRI datasets, the proposed algorithm achieved consistent dynamic needle feature tracking performance with processing time of 75 ms/image: (a) median dxy = 0.90 mm, median dθ = 1.53°; (b) median dxy = 1.31 mm, median dθ = 1.9°; (c) median dxy = 1.09 mm, median dθ = 0.91°.

Conclusions

The proposed algorithm using Mask R-CNN can accurately track the needle feature tip and axis on MR images from in vivo intra-procedural prostate biopsy cases and ex vivo real-time MRI experiments with a range of different conditions. The algorithm achieved pixel-level tracking accuracy in real time and has potential to assist MRI-guided percutaneous interventions.

previous article Navigated tissue characterization during skin cancer surgery

next article Automatic robot-world calibration in an optical-navigated surgical robot system and its application for oral implant placement

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

Gupta S, Madoff DC (2007) Image-guided percutaneous needle biopsy in cancer diagnosis and staging. Tech Vasc Interv Radiol 10:88–101. https://doi.org/10.1053/j.tvir.2007.09.005CrossRefPubMed

McWilliams JP, Lee EW, Yamamoto S, Loh CT, Kee ST (2010) Image-guided tumor ablation: emerging technologies and future directions. Semin Interv Radiol 27:302–313. https://doi.org/10.1055/s-0030-1261789CrossRef

Campbell-Washburn AE, Faranesh AZ, Lederman RJ, Hansen MS (2015) Magnetic resonance sequences and rapid acquisition for MR-guided interventions. Magn Reson Imaging Clin 23:669–679. https://doi.org/10.1016/j.mric.2015.05.006CrossRef

Campbell-Washburn AE, Tavallaei MA, Pop M, Grant EK, Chubb H, Rhode K, Wright GA (2017) Real-time MRI guidance of cardiac interventions. J Magn Reson Imaging 46:935–950. https://doi.org/10.1002/jmri.25749CrossRefPubMedPubMedCentral

DiMaio SP, Kacher D, Ellis R, Fichtinger G, Hata N, Zientara G, Panych L, Kikinis R, Jolesz F (2006) Needle artifact localization in 3T MR images. Stud Health Technol Inform 119:120–125PubMed

DiMaio SP, Samset E, Fischer G, Iordachita I, Fichtinger G, Jolesz F, Tempany CM (2007) Dynamic MRI scan plane control for passive tracking of instruments and devices. In: Medical image computing and computer-assisted intervention (MICCAI), pp 50–58. https://doi.org/10.1007/978-3-540-75759-7_7

Görlitz RA, Tokuda J, Hoge SW, Chu R, Panych LP, Tempany C, Hata N (2010) Development and validation of a real-time reduced field of view imaging driven by automated needle detection for MRI-guided interventions. In: SPIE medical imaging, pp 762515–762519. https://doi.org/10.1117/12.840837

Zijlstra F, Bouwman JG, Braškutė I, Viergever MA, Seevinck PR (2016) Fast Fourier-based simulation of off-resonance artifacts in steady-state gradient echo MRI applied to metal object localization. Magn Reson Med 78:2035–2041. https://doi.org/10.1002/mrm.26556CrossRefPubMedPubMedCentral

Moore CM, Robertson NL, Arsanious N, Middleton T, Villers A, Klotz L, Taneja SS, Emberton M (2013) Image-guided prostate biopsy using magnetic resonance imaging–derived targets: a systematic review. Eur Urol 63:125–140. https://doi.org/10.1016/j.eururo.2012.06.004CrossRefPubMed

10.

Tan N, Lin W-C, Khoshnoodi P, Asvadi NH, Yoshida J, Margolis DJA, Lu DSK, Wu H, Sung KH, Lu DY, Huang J, Raman SS (2016) In-bore 3-T MR-guided transrectal targeted prostate biopsy: prostate imaging reporting and data system version 2–based diagnostic performance for detection of prostate cancer. Radiology 283:130–139. https://doi.org/10.1148/radiol.2016152827CrossRefPubMedPubMedCentral

11.

Verma S, Choyke PL, Eberhardt SC, Oto A, Tempany CM, Turkbey B, Rosenkrantz AB (2017) The current state of MR imaging–targeted biopsy techniques for detection of prostate cancer. Radiology 285:343–356. https://doi.org/10.1148/radiol.2017161684CrossRefPubMedPubMedCentral

12.

Ladd ME, Erhart P, Debatin JF, Romanowski BJ, Boesiger P, McKinnon GC (1996) Biopsy needle susceptibility artifacts. Magn Reson Med 36:646–651. https://doi.org/10.1002/mrm.1910360423CrossRefPubMed

13.

Cole GA, Harrington K, Su H, Camilo A, Pilitsis JG, Fischer GS (2014) Closed-loop actuated surgical system utilizing real-time in situ MRI guidance. In: International symposium on experimental robotics (ISER), pp 785–798. https://doi.org/10.1007/978-3-642-28572-1_54

14.

Mikaiel S, Simonelli J, Li X, Lee Y, Lee YS, Sung K, Lu D, Tsao TC, Wu HH (2020) MRI-guided targeted needle placement during motion using hydrostatic actuators. Int J Med Robot Comput Assist Surg 16(2):e2041. https://doi.org/10.1002/rcs.2041CrossRef

15.

Mehrtash A, Ghafoorian M, Pernelle G, Ziaei A, Heslinga FG, Tuncali K, Fedorov A, Kikinis R, Tempany CM, Wells WM, Abolmaesumi P, Kapur T (2019) Automatic needle segmentation and localization in MRI with 3D convolutional neural networks: application to MRI-targeted prostate biopsy. IEEE Trans Med Imaging 38:1026–1036. https://doi.org/10.1109/TMI.2018.2876796CrossRefPubMed

16.

Zijlstra F, Viergever MA, Seevinck PR (2019) SMART tracking: simultaneous anatomical imaging and real-time passive device tracking for MR-guided interventions. Phys Med 64:252–260. https://doi.org/10.1016/j.ejmp.2019.07.019CrossRefPubMed

17.

Ho M, Kim Y, Cheng SS, Gullapalli R, Desai JP (2015) Design, development, and evaluation of an MRI-guided SMA spring-actuated neurosurgical robot. Int J Rob Res 34:1147–1163. https://doi.org/10.1177/0278364915579069CrossRefPubMedPubMedCentral

18.

Bergeles C, Vartholomeos P, Qin L, Dupont PE (2013) Closed-loop commutation control of an MRI-powered robot actuator. In: 2013 IEEE international conference on robotics and automation, pp 698–703. https://doi.org/10.1109/ICRA.2013.6630649

19.

Moreira P, Patel N, Wartenberg M, Li G, Tuncali K, Heffter T, Burdette EC, Iordachita I, Fischer GS, Hata N, Tempany CM, Tokuda J (2018) Evaluation of robot-assisted MRI-guided prostate biopsy: needle path analysis during clinical trials. Phys Med Biol 63:20NT02. https://doi.org/10.1088/1361-6560/aae214CrossRefPubMedPubMedCentral

20.

Mwikirize C, Nosher JL, Hacihaliloglu I (2018) Signal attenuation maps for needle enhancement and localization in 2D ultrasound. Int J Comput Assist Radiol Surg 13:363–374. https://doi.org/10.1007/s11548-017-1698-yCrossRefPubMed

21.

Uherčík M, Kybic J, Liebgott H, Cachard C (2010) Model fitting using RANSAC for surgical tool localization in 3-D ultrasound images. IEEE Trans Biomed Eng 57:1907–1916. https://doi.org/10.1109/TBME.2010.2046416CrossRefPubMed

22.

Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Cambridge

23.

Pourtaherian A, Ghazvinian Zanjani F, Zinger S, Mihajlovic N, Ng G, Korsten H, de With P (2017) Improving needle detection in 3D ultrasound using orthogonal-plane convolutional networks. In: Medical image computing and computer-assisted intervention (MICCAI), pp 610–618. https://doi.org/10.1007/978-3-319-66185-8_69

24.

Weine J, Breton E, Garnon J, Gangi A, Maier F (2019) Deep learning based needle localization on real-time MR images of patients acquired during MR-guided percutaneous interventions. In: Proceedings of the ISMRM 27th annual meeting, p 973

25.

He K, Gkioxari G, Dollár P, Girshick RB (2017) Mask R-CNN. In: International conference on computer vision (ICCV), pp 2980–2988. https://doi.org/10.1109/ICCV.2017.322

26.

Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: International conference on neural information processing systems (NIPS), pp 91–99. https://doi.org/10.1109/TPAMI.2016.2577031

27.

Vuola AO, Akram SU, Kannala J (2019) Mask-RCNN and U-Net ensembled for nuclei segmentation. In: IEEE international conference on biomedical imaging (ISBI), pp 208–212. https://doi.org/10.1109/isbi.2019.8759574

28.

Li X, Raman SS, Lu D, Lee Y, Tsao T, Wu HH (2019) Real-time needle detection and segmentation using Mask R-CNN for MRI-guided interventions. In: Proceedings of the ISMRM 27th annual meeting, p 972

29.

Abdulla W (2017) Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow. GitHub repository

30.

Li X, Lee Y-H, Mikaiel S, Simonelli J, Tsao TC, Wu HH (2020) Respiratory motion prediction using fusion-based multi-rate Kalman filtering and real-time golden-angle radial MRI. IEEE Trans Biomed Eng 67(6):1727–1738. https://doi.org/10.1109/TBME.2019.2944803CrossRefPubMed

31.

Hansen MS, Sørensen TS (2013) Gadgetron: an open source framework for medical image reconstruction. Magn Reson Med 69:1768–1776. https://doi.org/10.1002/mrm.24389CrossRefPubMed

32.

Sørensen TS, Atkinson D, Schaeffter T, Hansen MS (2009) Real-time reconstruction of sensitivity encoded radial magnetic resonance imaging using a graphics processing unit. IEEE Trans Med Imaging 28:1974–1985. https://doi.org/10.1109/TMI.2009.2027118CrossRefPubMed

33.

Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft COCO: common objects in context. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) European conference on computer vision (ECCV). Springer, Cham, pp 740–755. https://doi.org/10.1007/978-3-319-10602-1_48CrossRef

34.

Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? In: International conference on neural information processing systems (NIPS), pp 3320–3328

35.

Boggs PT, Rogers JE (1990) Orthogonal distance regression. Contemp Math 112:183–194. https://doi.org/10.6028/nist.ir.89-4197CrossRef

36.

Patel NA, van Katwijk T, Gang Li, Moreira P, Weijian Shang, Misra S, Fischer GS (2015) Closed-loop asymmetric-tip needle steering under continuous intraoperative MRI guidance. In: IEEE engineering in medicine and biology society (EMBC), pp 4869–4874. https://doi.org/10.1109/EMBC.2015.7319484

37.

Kim YK, Kim YK, Park HJ, Park MJ, Lee WJ, Choi D (2014) Noncontrast MRI with diffusion-weighted imaging as the sole imaging modality for detecting liver malignancy in patients with high risk for hepatocellular carcinoma. Magn Reson Imaging 32:610–618. https://doi.org/10.1016/j.mri.2013.12.021CrossRefPubMed

38.

Stamey TA, Freiha FS, McNeal JE, Redwine EA, Whittemore AS, Schmid HP (1993) Localized prostate cancer. Relationship of tumor volume to clinical significance for treatment of prostate cancer. Cancer 71:933–938. https://doi.org/10.1002/1097-0142(19930201)71:3+%3c933:aid-cncr2820711408%3e3.0.co;2-lCrossRefPubMed

39.

Renfrew M, Griswold M, Çavusoglu MC (2018) Active localization and tracking of needle and target in robotic image-guided intervention systems. Auton Robots 42:83–97. https://doi.org/10.1007/s10514-017-9640-2CrossRefPubMed

40.

Song S-E, Cho NB, Iordachita II, Guion P, Fichtinger G, Whitcomb LL (2011) A study of needle image artifact localization in confirmation imaging of MRI-guided robotic prostate biopsy. In: IEEE international conference on robotics and automation (ICRA), pp 4834–4839. https://doi.org/10.1109/ICRA.2011.5980309

Title: Automatic needle tracking using Mask R-CNN for MRI-guided percutaneous interventions
Authors: Xinzhou Li
Adam S. Young
Steven S. Raman
David S. Lu
Yu-Hsiu Lee
Tsu-Chin Tsao
Holden H. Wu
Publication date: 16-07-2020
Publisher: Springer International Publishing
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 10/2020
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-020-02226-8

Springer Professional

Abstract

Purpose

Methods

Results

Conclusions

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 10/2020

A surgical navigated cutting guide for mandibular osteotomies: accuracy and reproducibility of an image-guided mandibular osteotomy

Electromagnetic (EM) catheter path tracking in ultrasound-guided brachytherapy of the breast

A visual odometry base-tracking system for intraoperative C-arm guidance

Atlas-based liver segmentation for nonhuman primate research

Evaluation of long-term stability of monolithic 3D-printed robotic manipulator structures for minimally invasive surgery

Custom mastoid-fitting templates to improve cochlear implant electrode insertion trajectory

Premium Partner