Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

04-07-2021 | Original Article

Visual Localisation for Knee Arthroscopy

Authors: Artur Banach, Mario Strydom, Anjali Jaiprakash, Gustavo Carneiro, Anders Eriksson, Ross Crawford, Aaron McFadyen

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 12/2021

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Purpose

Navigation in visually complex endoscopic environments requires an accurate and robust localisation system. This paper presents the single image deep learning based camera localisation method for orthopedic surgery.

Methods

The approach combines image information, deep learning techniques and bone-tracking data to estimate camera poses relative to the bone-markers. We have collected one arthroscopic video sequence for four knee flexion angles, per synthetic phantom knee model and a cadaveric knee-joint.

Results

Experimental results are shown for both a synthetic knee model and a cadaveric knee-joint with mean localisation errors of 9.66mm/0.85\(^\circ \) and 9.94mm/1.13\(^\circ \) achieved respectively. We have found no correlation between localisation errors achieved on synthetic and cadaveric images, and hence we predict that arthroscopic image artifacts play a minor role in camera pose estimation compared to constraints introduced by the presented setup. We have discovered that the images acquired for 90°and 0°knee flexion angles are respectively most and least informative for visual localisation.

Conclusion

The performed study shows deep learning performs well in visually challenging, feature-poor, knee arthroscopy environments, which suggests such techniques can bring further improvements to localisation in Minimally Invasive Surgery.

previous article Fully automated deep learning-based localization and segmentation of the locus coeruleus in aging and Parkinson’s disease using neuromelanin-sensitive MRI

next article Intelligent optical diagnosis and treatment system for automated image-guided laser ablation of tumors

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Jaiprakash A, O’Callaghan WB, Whitehouse SL, Pandey A, Wu L, Roberts J, Crawford R (2017) Orthopaedic surgeon attitudes towards current limitations and the potential for robotic and technological innovation in arthroscopic surgery. J Orthop Surg 25(1):1–6

Banach A, Leibrandt K, Grammatikopoulou M, Yang G-Z (2019) Active contraints for tool-shaft collision avoidance in minimally invasive surgery. In: IEEE ICRA, pp 1556–1562

Marmol A, Corke P, Peynot T (2018) ArthroSLAM : multi-sensor robust visual localization for minimally invasive orthopedic surgery. In: IEEE IROS

Bailey T, Durrant-Whyte H (2006) Simultaneous localization and mapping (SLAM): part I. IEEE Robot Autom Mag 13(3):108–117CrossRef

Bailey T, Durrant-whyte H (2006) Simultaneous localization and mapping (SLAM): part II. IEEE Robot Autom Soc 13(September):108–117CrossRef

Cadena C, Carlone L, Carrillo H, Latif Y, Scaramuzza D, Reid I, Leonard JJ, Neira J, Reid I, Leonard JJ (2016) Past, present, and future of simultaneous localization and mapping: toward the robust-perception age. IEEE Trans 32(6):1309–1332

Maier-Hein L, Mountney P, Bartoli A, Elhawary H, Elson DS, Groch A, Kolb A, Rodrigues M, Sorger J, Speidel S, Stoyanov D (2013) Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery. Med Image Anal 17(8):974–996CrossRef

Stoyanov D, Yang G-Z (2009) Soft tissue deformation tracking for robotic assisted minimally invasive surgery. In: Annual international conference of the IEEE engineering in medicine and biology society, pp. 254–257, 2009

Mahmoud N, Grasa ÓG, Nicolau SA, Doignon C, Soler L, Marescaux J, Montiel JM (2017) On-patient see-through augmented reality based on visual SLAM. Int J Comput Assist Radiol Surg 12(1):1–11CrossRef

10.

Strydom M, Crawford R, Roberts J, Jaiprakash A (2019) Robotic arthroscopy: the uncertainty in internal knee joint measurement. IEEE Access 7:168382–168394CrossRef

11.

Marmol A, Peynot T, Eriksson A, Jaiprakash A, Roberts J, Crawford R (2017) Evaluation of keypoint detectors and descriptors in arthroscopic images for feature-based matching applications. IEEE Robot Autom Lett 2(4):2135–2142CrossRef

12.

Marmol A, Banach A, Peynot T (2019) Dense-ArthroSLAM: dense intra-articular 3-D reconstruction with robust localization prior for arthroscopy. IEEE Robot Autom Lett 4(2):918–925CrossRef

13.

Mur-Artal R, Tardós JD (2017) Orb-slam2: an open-source slam system for monocular, stereo, and rgb-d cameras. IEEE Trans Robot 33(5):1255–1262CrossRef

14.

Engel J, Schops T, Cremers D (2014) LSD-SLAM: large-scale direct monocular SLAM. In: European conference on computer vision, pp 834–849

15.

Banach A, Strydom M, Jaiprakash A, Carneiro G, Brown C, Crawford R, McFadyen A (2020) Saliency improvement in feature-poor surgical environments using local Laplacian of Specified Histograms. IEEE Access 8:1–1CrossRef

16.

Goodfellow I, Bengio Y, Courville A (2016) Deep learning: whole book. Nature

17.

Mahmood F, Chen R, Durr NJ (2018) Unsupervised reverse domain adaptation for synthetic medical images via adversarial training. IEEE Trans Med Imaging 37(12):2572–2581CrossRef

18.

Visentini-scarzanella M, Koto S, Sugiura T, Kaneko T (2017) Deep monocular 3D reconstruction for assisted navigation in bronchoscopy. Int J Comput Assist Radiol Surg 7:1089–1099CrossRef

19.

Hou B, Alansary A, McDonagh S, Davidson A, Rutherford M, Hajnal JV, Rueckert, Glocker DB, Kainz B (2017) Predicting slice-to-volume transformation in presence of arbitrary subject motion. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 296–304

20.

Kausch L, Thomas S, Kunze H, Privalov M, Vetter S, Franke J, Mahnken AH, Maier-Hein L, Maier-Hein K (2020) Toward automatic c-arm positioning for standard projections in orthopedic surgery. Int J Comput Assist Radiol Surg 15(7):1095–1105CrossRef

21.

Sganga J, Eng D, Graetzel C, Camarillo D (2018) OffsetNet: deep learning for localization in the lung using rendered images. ArXiv e-prints

22.

Sganga J, Eng D, Graetzel C, Camarillo DB (2019) Deep learning for localization in the lung. ArXiv e-prints

23.

Zhou T, Berkeley UC, Brown M, Snavely N, Lowe DG (1851) Unsupervised learning of depth and ego-motion from video, pp 1851–1860

24.

Wang R, Pizer SM, Frahm JM (2019) Recurrent neural network for (un-)supervised learning of monocular video visual odometry and depth. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, 2019

25.

Kendall A, Cipolla R (2015) PoseNet: a convolutional network for real-time 6-DOF camera relocalization. In: International conference on computer vision, pp 2938–2946

26.

Strydom M, Banach A, Wu L, Jaiprakash A, Crawford R, Roberts J (2020) Anatomical joint measurement with application to medical robotics. IEEE Access

27.

Strydom ML, Banach A, Roberts J, Crawford R, Jaiprakash AT (2020) Kinematic model of the human leg using dh parameters. IEEE Access 8:191737–191750

28.

Wengert C, Reeff M, Cattin PC, Székely G (2006) Fully automatic endoscope calibration for intraoperative use. In: Bildverarbeitung für die Medizin 2006

29.

Bouguet J-Y (2002) Camera calibration toolbox for matlab

30.

Abadi M, Barham P, Chen J, Chen Z, Davis A, Jeffrey D, Devin M, Ghemawat S, Irving G, Isard M, Kudlur M, Levenberg V, Monga R, Moore S, Murray D, Steiner B, Tucker P, Vasudevan V, Warden P, Wicke M, Yu Y, Zheng X (2016) TensorFlow: a system for large-scale machine learning. In: Proceedings of the 12th USENIX conference on operator system design and implementation, 2016

31.

Mayo B, Hazan T, Tal A (2021) Visual navigation with spatial attention. arXiv preprint arXiv:2104.09807

32.

Shen M, Gu Y, Liu N, Yang GZ (2019) Context-aware depth and pose estimation for bronchoscopic navigation. IEEE Robot Autom Lett 4(2):732–739CrossRef

33.

Lu T-W, O’connor J (1999) Bone position estimation from skin marker co-ordinates using global optimisation with joint constraints. J Biomech 32(2):129–134CrossRef

34.

Charlton IW, Tate P, Smyth P, Roren L (2004) Repeatability of an optimised lower body model. Gait Posture 20(2):213–221CrossRef

35.

Carse B, Meadows B, Bowers R, Rowe P (2013) Affordable clinical gait analysis: an assessment of the marker tracking accuracy of a new low-cost optical 3D motion analysis system. Physiotherapy (United Kingdom) 99(4):347–351

36.

Allen MW, Jacofsky DJ (2019) Evolution of robotics in arthroplasty. In: Lonner JH (ed) Robotics in knee and hip arthroplasty. Springer, Berlin, pp 13–25CrossRef

Title: Visual Localisation for Knee Arthroscopy
Authors: Artur Banach
Mario Strydom
Anjali Jaiprakash
Gustavo Carneiro
Anders Eriksson
Ross Crawford
Aaron McFadyen
Publication date: 04-07-2021
Publisher: Springer International Publishing
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 12/2021
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-021-02444-8

Springer Professional

Abstract

Purpose

Methods

Results

Conclusion

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 12/2021

Intelligent optical diagnosis and treatment system for automated image-guided laser ablation of tumors

High-resolution deep transferred ASPPU-Net for nuclei segmentation of histopathology images

Versatile anomaly detection method for medical images with semi-supervised flow-based generative models

Force-guided autonomous robotic ultrasound scanning control method for soft uncertain environment

Detecting failure modes in image reconstructions with interval neural network uncertainty

Fully automated deep learning-based localization and segmentation of the locus coeruleus in aging and Parkinson’s disease using neuromelanin-sensitive MRI

Premium Partner