Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

01-08-2016 | Original Article

Precise 3D/2D calibration between a RGB-D sensor and a C-arm fluoroscope

Authors: Xiang Wang, Severine Habert, Meng Ma, Chun-Hao Huang, Pascal Fallavollita, Nassir Navab

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 8/2016

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Purpose

Calibration and registration are the first steps for augmented reality and mixed reality applications. In the medical field, the calibration between an RGB-D camera and a C-arm fluoroscope is a new topic which introduces challenges.

Method

A convenient and efficient calibration phantom is designed by combining the traditional calibration object of X-ray images with a checkerboard plane. After the localization of the 2D marker points in the X-ray images and the corresponding 3D points from the RGB-D images, we calculate the projection matrix from the RGB-D sensor coordinates to the X-ray, instead of estimating the extrinsic and intrinsic parameters simultaneously.

Validation

In order to evaluate the effect of every step of our calibration process, we performed five experiments by combining different steps leading to the calibration. We also compared our calibration method to Tsai’s method to evaluate the advancement of our solution. At last, we simulated the process of estimating the rotation movement of the RGB-D camera using MATLAB and demonstrate that calculating the projection matrix can reduce the angle error of the rotation.

Results

A RMS reprojection error of 0.5 mm is achieved using our calibration method which is promising for surgical applications. Our calibration method is more accurate when compared to Tsai’s method. Lastly, the simulation result shows that using a projection matrix has a lower error than using intrinsic and extrinsic parameters in the rotation estimation.

Conclusions

We designed and evaluated a 3D/2D calibration method for the combination of a RGB-D camera and a C-arm fluoroscope.

next article Assessment of image features for vessel wall segmentation in intravascular ultrasound images

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Kersten-Oertel M, Jannin P, Collins DL (2012) DVV: a taxonomy for mixed reality visualization in image guided surgery. IEEE Trans Vis Comput Graph 18(2):332–352CrossRefPubMed

Navab N, Blum T, Wang L, Okur A, Wendler T (2012) First deployments of augmented reality in operating rooms. Comput IEEE Comput Soc 7:48–55CrossRef

Ewers R, Schicho K, Wagner A, Undt G, Seemann R, Figl M, Truppe M (2005) Seven years of clinical experience with teleconsultation in craniomaxillofacial surgery. J Oral Maxillofac Surg 63:1447–1454CrossRefPubMed

Johnson LG, Edwards P, Hawkes D (2003) Surface transparency makes stereo overlays unpredictable: the implications for augmented reality. Stud Health Technol Inform 94:131–136

Bauer S, Seitel A, Hofmann H, Blum T, Wasza J, Balda M, Meinzer HP, Navab N, Hornegger J, Maier-Hein L (2013) Real-time range imaging in health care: a survey. In: Grzegorzek M, Theobalt C, Koch R, Kolb A (eds) Time-of-flight and depth imaging. Sensors, algorithms, and applications. Springer, Berlin, Heidelberg, pp 228–254

Wang XL, Stolka PJ, Boctor E, Hager G, Choti M (2012) The Kinect as an interventional tracking system. In: David R. Holmes III, Kenneth H. Wong (eds) SPIE medical imaging 2012: image-guided procedures, robotic interventions and modeling, vol 8316. SPIE, San Diego, California, USA, pp 83160U-83160U-6

Kontschieder P, Dorn JF, Morrison C, Corish R, Zikic D, Sellen A, D’Souza M, Kamm CP, Burggraaff J, Tewarie P, Vogel T, Azzarito M, Glocker B, Dahlke P, Polman C, Kappos L, Uitdehaag B, Criminisi A (2014) Quantifying progression of multiple sclerosis via classification of depth videos. In: Golland P, Hata N, Barillot C, Hornegger J, Howe R (eds) Medical image computing and computer-assisted intervention—MICCAI 2014, vol 8674. Springer International Publishing, Switzerland, pp 429–437

Alnowami M, Alnwaimi B, Tahavori F, Copland M, Wells K (2012) A quantitative assessment of using the Kinect for Xbox360 for respiratory surface motion tracking. In: David R. Holmes III, Kenneth H. Wong (eds) SPIE medical imaging, international society for optics and photonics, SPIE, San Diego, California, USA, pp 83161T–83161T-10

Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334CrossRef

10.

Gábor B (2015) Camera calibration with opencv.http://www.swarthmore.edu/NatSci/mzucker1/opencv-2.4.10-docs/doc/tutorials/calib3d/camera_calibration/camera_calibration.html#cameracalibrationopencv. Accessed 24 Nov 2015

11.

Yaniv Z, Joskowicz L, Simkin A, Garza-Jinich M, Milgrom C (1998) Fluoroscopic image processing for computer-aided orthopaedic surgery. In: Wells WM, Colchester A, Delp S (eds) Medical image computing and computer-assisted interventation—MICCAI’98, vol 1496. Springer, Berlin, Heidelberg, pp 325–334

12.

Livyatan H, Yaniv Z, Joskowicz L (2002) Robust automatic C-arm calibration for fluoroscopy-based navigation: a practical approach. In: Dohi T, Kikinis R (eds) Medical image computing and computer-assisted intervention–MICCAI 2002, vol 2489. Springer, Berlin, Heidelberg, pp 60–68

13.

Meng C, Zhang J, Zhou F, Wang T (2014) New method for geometric calibration and distortion correction of conventional C-arm. Comput Biol Med 52:49–56CrossRefPubMed

14.

Shechter G, Devernay F, Coste-Manière E, McVeigh ER (2002) Temporal tracking of 3D coronary arteries in projection angiograms. In: Medical imaging 2002, international society for optics and photonics. pp 612–623

15.

Shechter G, Shechter B, Resar JR, Beyar R (2005) Prospective motion correction of X-ray images for coronary interventions. IEEE Trans Med Imaging 24(4):441–450CrossRefPubMed

16.

Tsai R (1987) A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lens. IEEE J Robot Autom 3:323–344CrossRef

17.

Jung J, Lee JY, Jeong Y, Kweon IS (2014) Time-of-flight sensor calibration for a color and depth camera pair. IEEE Trans Pattern Anal Mach Intell 37(7):1501–1513CrossRef

18.

Shime H, Adelsberger R, Kim JD, Rhee SM, Rhee T, Sim JY, Gross M, Kim C (2012) Time-of-flight sensor and color camera calibration for multi-view acquisition. Vis Comput 28:1139–1151CrossRef

19.

Zhang C, Zhang Z (2014) Calibration between depth and color sensors for commodity depth cameras. In: Shao L, Han J, Kohli P, Zhang Z (eds) Computer vision and machine learning with RGB-D sensors. Springer International Publishing, Switzerland, pp 47–64

20.

Herrera C, Kannala J, Heikkilä J (2012) Joint depth and color camera calibration with distortion correction. IEEE Trans Pattern Anal Mach Intell 34(10):2058–8828CrossRef

21.

Raposo C, Barreto JP, Nunes U (2013) Fast and accurate calibration of a Kinect sensor. In: International conference on 3D vision—3DV. pp 342–349

22.

Horaud R, Dornaika F (1995) Hand-eye calibration. Int J Robot Res 14(3):195–210CrossRef

23.

Mitschke MM, Navab N (2000) Recovering projection geometry: how a cheap camera can outperform an expensive stereo system. In: IEEE conference on computer vision and pattern recognition (CVPR), vol. 1. pp 193–200

24.

Navab N, Bani-Hashemi A, Nadar MS, Wiesent K, Durlak P, Brunner T, Barth K, Graumann R (1998) 3D Reconstruction from projection matrices in a C-arm based 3D-angiography system. In: Wells WM, Colchester A, Delp S (eds) Medical image computing and computer assisted interventation—MICCAI’98, vol 1496. Springer, Berlin, Heidelberg, pp 119–129

25.

Yang L, Zhang L, Dong H, Alelaiwi A, Saddik AE (2015) Evaluating and improving the depth accuracy of Kinect for Windows v2. IEEE Sens J 15(8):4275–4285CrossRef

26.

Kedgley AE, Fox AV, Jenkyn TR (2012) Image intensifier distortion correction for fluoroscopic RSA: the need for independent accuracy assessment. J Appl Clin Med Phys 13(1):197–204

27.

Navab N, Mitschke M (2001) Method and apparatus using a virtual detector for three-dimensional reconstruction from X-ray images. Patent US 6236704; Filing date: June 30, 1999; Issue date: May 22

28.

Huhle B, Schairer T, Jenke P, Strasser W (2008) Robust non-local denosing of colored depth data. In: IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW’08). pp 1–7

29.

Essmaeel K, Gallo L, Damiani E, De Pietro G, Dipanda A (2012) Temporal denoising of Kinect depth data. In: 2012 Eighth international conference on signal image technology and internet based systems (SITIS). pp 47–52

30.

Hartley R, Zisserman A (2004) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, CambridgeCrossRef

31.

Moré JJ (1978) The Levenberg–Marquardt algorithm: implementation and theory. In: Watson GA (ed) Numerical analysis. Springer, Berlin, Heidelberg, pp 105–116

32.

Horn BKP (2000) Tsai’s camera calibration method revisited. http://people.csail.mit.edu/bkph/articles/Tsai_Revisited.pdf

33.

Golub GH, Van Loan CF (1996) Matrix computation, 3rd edn. Johns Hopkins University Press, Baltimore

34.

Horn BKP (2008) Finding the nearest orthonormal matrix. http://people.csail.mit.edu/bkph/articles/Nearest_Orthonormal_Matrix.pdf

35.

Huynh DQ (2009) Metrics for 3D rotation: comparison and analysis. J Math Imaging Vis 35:155–164CrossRef

36.

Pauly O, Diotte B, Fallavollita P, Weidert S, Euler E, Navab N (2015) Machine learning-based augmented reality for improved surgical scene understanding. Comput Med Imaging Graph 41:55–60CrossRefPubMed

37.

Pauly O, Diotte B, Habert S, Weidert S, Euler E, Fallavollita P, Navab N (2014). Relevance-based visualization to improve surgeon perception. In: Stoyanov D, Collins DL, Sakuma I, Abolmaesumi P, Jannin P (eds) Information processing in computer-assisted interventions, vol 8498. Springer International Publishing, Switzerland, pp 178–185

38.

Pauly O, Katouzian A, Eslami A, Fallavollita P, Navab N (2012). Supervised classification for customized intraoperative augmented reality visualization. In: 2012 IEEE international symposium on mixed and augmented reality (ISMAR). IEEE, pp 311–312

39.

Wieczorek M, Aichert A, Fallavollita P, Kutter O, Ahmadi A, Wang L, Navab N (2011) Interactive 3D visualization of a single-view X-ray image. In: Fichtinger G, Martel A, Peters T (eds) Medical image computing and computer-assisted intervention-MICCAI 2011, vol 6891. Springer, Berlin, Heidelberg, pp 73–80

40.

Erat O, Pauly O, Weidert S, Thaller P, Euler E, Mutschler W, et al (2013) How a surgeon becomes superman by visualization of intelligently fused multi-modalities. In: David R. Holmes, Ziv R. Yaniv (eds) SPIE medical imaging. International Society for Optics and Photonics, vol 8671. SPIE, Florida, USA, pp 86710L–86710L-6

41.

Wang L, Fallavollita P, Brand A, Erat O, Weidert S, Thaller PH et al (2012) Intra-op measurement of the mechanical axis deviation: an evaluation study on 19 human cadaver legs. In: Ayache N, Delingette H, Golland P, Mori K (eds) Medical image computing and computer-assisted intervention—MICCAI 2012. Springer, Berlin, Heidelberg, pp 609–616

42.

Diotte B, Fallavollita P, Wang L, Weidert S, Euler E, Thaller P, Navab N (2015) Multi-modal intra-operative navigation during distal locking of intramedullary nails. Med Imaging IEEE Trans 34(2):487–495CrossRef

43.

Londei R, Esposito M, Diotte B, Weidert S, Euler E, Thaller P, et al (2014) The ‘augmented’circles: a video-guided solution for the down-the-beam positioning of IM nail holes. In: Stoyanov D, Collins DL, Sakuma I, Abolmaesumi P, Jannin P (eds) Information processing in computer-assisted interventions, vol 8498. Springer International Publishing, Switzerland, pp 100–107

44.

Londei R, Esposito M, Diotte B, Weidert S, Euler E, Thaller P, et al (2015) Intra-operative augmented reality in distal locking. Int J Comput Assist Radiol Surg 10(9):1395–1403

Title: Precise 3D/2D calibration between a RGB-D sensor and a C-arm fluoroscope
Authors: Xiang Wang
Severine Habert
Meng Ma
Chun-Hao Huang
Pascal Fallavollita
Nassir Navab
Publication date: 01-08-2016
Publisher: Springer Berlin Heidelberg
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 8/2016
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-015-1347-2

Springer Professional

Abstract

Purpose

Method

Validation

Results

Conclusions

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 8/2016

Comparing position and orientation accuracy of different electromagnetic sensors for tracking during interventions

Magnetic resonance imaging compatible remote catheter navigation system with 3 degrees of freedom

Endoscopic scene labelling and augmentation using intraoperative pulsatile motion and colour appearance cues with preoperative anatomical priors

On computerized methods for spine analysis in MRI: a systematic review

An open tool for input function estimation and quantification of dynamic PET FDG brain scans

Design and evaluation of an interactive training system for scrub nurses

Premium Partner