Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
International Journal of Computer Assisted Radiology and Surgery
Published in: International Journal of Computer Assisted Radiology and Surgery 1/2017

04-07-2016 | Original Article

On-patient see-through augmented reality based on visual SLAM

Authors: Nader Mahmoud, Óscar G. Grasa, Stéphane A. Nicolau, Christophe Doignon, Luc Soler, Jacques Marescaux, J. M. M. Montiel

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 1/2017

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Purpose

An augmented reality system to visualize a 3D preoperative anatomical model on intra-operative patient is proposed. The hardware requirement is commercial tablet-PC equipped with a camera. Thus, no external tracking device nor artificial landmarks on the patient are required.

Methods

We resort to visual SLAM to provide markerless real-time tablet-PC camera location with respect to the patient. The preoperative model is registered with respect to the patient through 4–6 anchor points. The anchors correspond to anatomical references selected on the tablet-PC screen at the beginning of the procedure.

Results

Accurate and real-time preoperative model alignment (approximately 5-mm mean FRE and TRE) was achieved, even when anchors were not visible in the current field of view. The system has been experimentally validated on human volunteers, in vivo pigs and a phantom.

Conclusions

The proposed system can be smoothly integrated into the surgical workflow because it: (1) operates in real time, (2) requires minimal additional hardware only a tablet-PC with camera, (3) is robust to occlusion, (4) requires minimal interaction from the medical staff.
next article The production of digital and printed resources from multiple modalities using visualization and three-dimensional printing techniques

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
Appendix
Available only for authorised users
Literature
1.
Hallet J, Soler L, Diana M, Mutter D, Baumert TF, Habersetzer F, Marescaux J, Pessaux P (2015) Trans-thoracic minimally invasive liverresection guided by augmented reality. J AmColl Surgeons 220(5):e55e60
2.
Kilgus T, Heim E, Haase S, Prufer S, Muller M, Seitel A, Fangerau M, Wiebe T, Iszatt J, Schlemmer HP, Hornegger J, Yen K, Maier-Hein L (2015) Mobile markerless augmented reality and its application in forensic medicine. IJCARS 10(5):573–586
3.
dos Santos T, Seitel A, Kilgus T, Suwelack S, Wekerle AL, Kenngott H, Speidel S, Schlemmer HP, Meinzer HP, Heimann T, Maier-Hein L (2014) Pose-independent surface matching for intra-operative soft-tissue marker-less registration. Med Image Anal 18(7):1101–1114CrossRefPubMed
4.
Macedo M, Souza A, Giraldi G (2014) High-quality on-patient medical data visualization in a markerless augmented reality environment. SBC J Interact Syst 5(3):41–52
5.
Lee J, Huang C, Huang T, Hsieh H, Lee S (2012) Medical augment reality using a markerless registration framework. Int J Expert Syst Appl 39(5):5286–5294CrossRef
6.
Chen X, Xu L, Wang Y, Wang H, Wang F, Zeng X, Wangb Q, Egger J (2015) Development of a surgical navigation system based on augmented reality using an optical see-through head-mounted display. J Biomed Inform 55:124–131CrossRefPubMed
7.
Rassweiler JJ, Müller M, Fangerau M, Klein J, Goezen AS, Pereirac P, Meinzerb HP, Teberd D (2012) iPad-assisted percutaneous access to the kidney using marker-based navigation: initial clinical experience. Eur Urol 61(3):628–631CrossRefPubMed
8.
Muller M, Rassweiler M, Klein J, Seitel A, Gondan M, Baumhauer M, Teber G, Rassweiler JJ, Meinzer HP, Maier-Hein L (2013) Mobile augmented reality for computer-assisted percutaneous nephrolithotomy. IJCARS 8(4):663–675
9.
Sun Y, Luebbers H, Agbaje J, Schepers S, Vrielinck L, Lambrichts I, Politis C (2013) Validation of anatomical landmarks-based registration for image-guided surgery: an in-vitro study. J Cranio Maxill Surg 41(6):522–526CrossRef
10.
Schneider A, Baumberger C, Griessen M, Pezold S, Beinemann J, Philipp Jurgens P, Cattin PC (2014) Landmark-based surgical navigation. Clinical image-based procedures. Transl Res Med Imaging 8361:57–64
11.
Davison AJ (2003) Real-time simultaneous localisation and mapping with a single camera. IEEE Int Conf Comput Vis 2:1403–1410
12.
Civera J, Davison AJ, Montiel JMM (2008) Inverse depth parametrization for monocular SLAM. IEEE Trans Robot 24(5):932–945CrossRef
13.
Civera J, Grasa OG, Davison AJ, Montiel JMM (2010) 1-Point RANSAC for extended Kalman filtering. Application to real time structure from motion and visual odometry. J Field Robot 27(5):609–631CrossRef
14.
Grasa OG, Bernal E, Casado S, Gil I, Montiel JMM (2014) Visual SLAM for handheld monocular endoscope. IEEE Trans Med Imaging 33(1):135–146CrossRefPubMed
15.
Grasa OG, Civera J, Montiel JMM (2009) EKF monocular SLAM 3D modeling. Measuring and augmented reality from endoscope image sequences. In MICCAI, vol 2
16.
Klein G, Murray D (2007) Parallel tracking and mapping for small AR workspace. In: IEEE and ACM international symposium on mixed and augmented reality (ISMAR), pp 1–10
17.
Mikhail EM, Bethel JS, McGlone JC (2001) Introduction to modern photogrammetry. Wiley, New York
18.
Mur-Artal R, Montiel JMM, Tard JD (2015) ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans Robot 31(5):1147–1163CrossRef
19.
Rublee E, Rabaud V, Konolige K, Bradski G (2011) ORB: an efficient alternative to SIFT or SURF. In: IEEE international conference on computer vision (ICCV), pp 2564–2571
20.
Lowe DG (1999) Object recognition from local scale-invariant features. In: Proceedings of IEEE international conference on computer vision, vol 2, pp 1150–1157
21.
Rosten E, Drummond T (2006) Machine learning for high-speed corner detection. In: Proceedings of 9th European conference on computer vision, pp 430–443
22.
Shi J, Tomasi C (1994) Good features to track. In: IEEE computer society conference in computer vision and pattern recognition, pp 593–600
23.
Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110CrossRef
24.
25.
Gao X, Hou X, Tang J, Cheng H (2003) Complete solution classification for the perspective-three-point problem. IEEE Trans Pattern Anal 25(8):930–943CrossRef
26.
27.
Horn BK (1987) Closed-form solution of absolute orientation using unit quaternions. J Opt Soc Am A 4(4):629–642CrossRef
28.
Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal 22(11):1330–1334CrossRef
29.
Metadata
Title
On-patient see-through augmented reality based on visual SLAM
Authors
Nader Mahmoud
Óscar G. Grasa
Stéphane A. Nicolau
Christophe Doignon
Luc Soler
Jacques Marescaux
J. M. M. Montiel
Publication date
04-07-2016
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 1/2017
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-016-1444-x

Other articles of this Issue 1/2017

International Journal of Computer Assisted Radiology and Surgery 1/2017 Go to the issue

Original Article

Crowdsourcing for error detection in cortical surface delineations

Original Article

Robust colonoscope tracking method for colon deformations utilizing coarse-to-fine correspondence findings

Original Article

Robotic ultrasound-guided SBRT of the prostate: feasibility with respect to plan quality

Original Article

Planning of mandibular reconstructions based on statistical shape models

Review Article

2D versus 3D fluoroscopy-based navigation in posterior pelvic fixation: review of the literature on current technology

Original Article

Shared control of a medical robot with haptic guidance

Premium Partner

    Image Credits