nach oben

International Journal of Computer Assisted Radiology and Surgery

Erschienen in:

01.05.2015 | Original Article

Mobile markerless augmented reality and its application in forensic medicine

verfasst von: Thomas Kilgus, Eric Heim, Sven Haase, Sabine Prüfer, Michael Müller, Alexander Seitel, Markus Fangerau, Tamara Wiebe, Justin Iszatt, Heinz-Peter Schlemmer, Joachim Hornegger, Kathrin Yen, Lena Maier-Hein

Erschienen in: International Journal of Computer Assisted Radiology and Surgery | Ausgabe 5/2015

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Purpose

During autopsy, forensic pathologists today mostly rely on visible indication, tactile perception and experience to determine the cause of death. Although computed tomography (CT) data is often available for the bodies under examination, these data are rarely used due to the lack of radiological workstations in the pathological suite. The data may prevent the forensic pathologist from damaging evidence by allowing him to associate, for example, external wounds to internal injuries. To facilitate this, we propose a new multimodal approach for intuitive visualization of forensic data and evaluate its feasibility.

Methods

A range camera is mounted on a tablet computer and positioned in a way such that the camera simultaneously captures depth and color information of the body. A server estimates the camera pose based on surface registration of CT and depth data to allow for augmented reality visualization of the internal anatomy directly on the tablet. Additionally, projection of color information onto the CT surface is implemented.

Results

We validated the system in a postmortem pilot study using fiducials attached to the skin for quantification of a mean target registration error of \(4.4 \pm 1.3\) mm.

Conclusions

The system is mobile, markerless, intuitive and real-time capable with sufficient accuracy. It can support the forensic pathologist during autopsy with augmented reality and textured surfaces. Furthermore, the system enables multimodal documentation for presentation in court. Despite its preliminary prototype status, it has high potential due to its low price and simplicity.

Vorheriger Artikel Multi-slice-to-volume registration for MRI-guided transperineal prostate biopsy

Nächster Artikel Influence of angulation on metacarpal bone mineral density measurements using digital X-ray radiogrammetry

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Nur mit Berechtigung zugänglich

E.g., in the institute for Forensic Medicine and Traffic Medicine in Heidelberg, almost for every case a CT scan is acquired before autopsy.

http://www.mitk.org.

We provide design and 3D printing plan publicly at http://www.thingiverse.com/thing:343425.

http://www.microsoft.com/en-us/kinectforwindowsdev/.

http://www.openni.org.

Bronchoscopy Model Sick Boy http://www.gtsimulators.com.

http://structure.io.

http://www.xbox.com/en-US/xbox-one/get-the-facts?xr=shellnav.

Ampanozi G, Zimmermann D, Hatch GM, Ruder TD, Ross S, Flach PM, Thali MJ, Ebert LC (2012) Format preferences of district attorneys for post-mortem medical imaging reports: understandability, cost effectiveness, and suitability for the courtroom: A questionnaire based study. Legal Med 14(3):116–120CrossRefPubMed

Batcher KE (1968) Sorting networks and their applications. In: Proceedings of the April 30-May 2, Spring Joint Computer Conference, pp 307–314

Bauer S, Seitel A, Hofmann H, Blum T, Wasza J, Balda M, Meinzer HP, Navab N, Hornegger J, Maier-Hein L (2013) Real-time range imaging in health care: a survey. In: Time-of-flight and depth imaging. Sensors, algorithms, and applications. Springer, Berlin, pp 228–254

Besl PJ, Neil DM (1992) A method for registration of 3-d shapes. IEEE Trans Pattern Anal Mach Intell 14(2):239–256CrossRef

Bhatia A, Patel S, Pantol G, Wu YY, Plitnikas M, Hancock C (2013) Intra and inter-observer reliability of mobile tablet PACS viewer system versus standard PACS viewing station-diagnosis of acute central nervous system events. Open J Radiol 03(02):91–98CrossRef

Blum T, Kleeberger V, Bichlmeier C, Navab N (2012) mirracle: an augmented reality magic mirror system for anatomy education. In: Virtual reality short papers and posters (VRW), IEEE, pp 115–116

Chetverikov D, Svirko D, Stepanov D, Krsek P (2002) The trimmed iterative closest point algorithm. In: Proceedings of the 16th international conference on pattern recognition, vol 3. IEEE, pp 545–548

Dabrowski JR, Munson EV (2001) Is 100 milliseconds too fast? In: CHI EA ’01 CHI ’01 Extended abstracts on human factors in computing systems. ACM Press, pp 317–318

Danilchenko A, Fitzpatrick JM (2011) General approach to first-order error prediction in rigid point registration. IEEE Trans Med Imaging 30(3):679–693CrossRefPubMed

10.

dos Santos TR, Seitel A, Kilgus T, Suwelack S, Wekerle AL, Kenngott H, Speidel S, Schlemmer HP, Meinzer HP, Heimann T, Maier-Hein L (2014) Pose-independent surface matching for intra-operative soft-tissue marker-less registration. Med Image Anal (accepted subject to minor revisions)

11.

Ebert L, Hatch G, Thali M, Ross S (2013) Invisible touch–control of a DICOM viewer with finger gestures using the kinect depth camera. J Forensic Radiol Imaging 1(1):10–14CrossRef

12.

Ebert LC, Hatch G, Ampanozi G, Thali MJ, Ross S (2012) You can’t touch this: touch-free navigation through radiological images. Surg Innov 19(3):301–307CrossRefPubMed

13.

Ebert LC, Ptacek W, Breitbeck R, Fürst M, Kronreif G, Martinez RM, Thali M, Flach PM (2014) Virtobot 2.0: the future of automated surface documentation and CT-guided needle placement in forensic medicine. Forensic Sci Med Pathol 10(2):179–186CrossRefPubMed

14.

Ebert LC, Ptacek W, Naether S, Fürst M, Ross S, Buck U, Weber S, Thali M (2009) Virtobot: a multi-functional robotic system for 3D surface scanning and automatic post mortem biopsy. Int J Med Robot Comput Assist Surg 6:18–27

15.

Glocker B, Izadi S, Shotton J, Criminisi A (2013) Real-time RGB-D camera relocalization. In: IEEE international symposium on mixed and augmented reality (ISMAR). IEEE, pp 173–179

16.

Handa A, Newcombe RA, Angeli A, Davison AJ (2012) Real-time camera tracking: when is high frame-rate best? In: Computer vision-ECCV 2012. Springer, pp 222–235

17.

Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press, Cambridge

18.

Horn BKP (1987) Closed-form solution of absolute orientation using unit quaternions. J Opt Soc Am A 4(4):629–642CrossRef

19.

Izadi S, Kim D, Hilliges O, Molyneaux D, Newcombe R, Kohli P, Shotton J, Hodges S, Freeman D, Davison A, Fritzgibbon A (2011) KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In: Proceedings of the 24th annual ACM symposium on user interface software and technology. ACM, pp 559–568

20.

John S, Poh ACC, Lim TCC, Chan EHY, Chong LR (2012) The iPad tablet computer for mobile on-call radiology diagnosis? Auditing discrepancy in CT and MRI reporting. J Digit Imaging 25(5):628–634CrossRefPubMedCentralPubMed

21.

Khoshelham K, Elberink SO (2012) Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(12):1437–1454CrossRefPubMedCentralPubMed

22.

Kilgus T, Franz A, Seitel A, März K, Bartha L, Fangerau M, Mersmann S, Groch A, Meinzer HP, Maier-Hein L (2012) Registration of partially overlapping surfaces for range image based augmented reality on mobile devices. In: SPIE medical imaging. International Society for Optics and Photonics, pp 83160T–83160T

23.

Kilgus T, dos Santos TR, Seitel A, Yung K, Franz AM, Groch A, Wolf I, Meinzer HP, Maier-Hein L (2011) Generation of triangle meshes from time-of-flight data for surface registration. In: Bildverarbeitung für die Medizin 2011. Springer, pp 189–193

24.

Kirk DB, Wen-mei WH (2012) Programming massively parallel processors: a hands-on approach. Newnes

25.

Kramer J, Burrus N, Echtler F, Daniel HC, Parker M (2012) Hacking the Kinect. Springer, New YorkCrossRef

26.

Li H, Sumner RW, Pauly M (2008) Global correspondence optimization for non-rigid registration of depth scans. In: Eurographics symposium on geometry processing, vol 27. Wiley, pp 1421–1430

27.

Lorensen WE, Cline HE (1987) Marching cubes: a high resolution 3d surface construction algorithm. In: ACM siggraph computer graphics, vol 21. ACM, pp 163–169

28.

Maier-Hein L, Fangerau M, Meinzer HP, Seitel A (2013) Visualization of anatomical data by augmented reality. US Patent App. 13/891,310

29.

Maier-Hein L, Franz AM, Fangerau M, Schmidt M, Seitel A, Mersmann S, Kilgus T, Groch A, Yung K, dos Santos TR, Meinzer HP (2011) Towards mobile augmented reality for on-patient visualization of medical images. In: Bildverarbeitung für die Medizin 2011, Informatik Aktuell. Springer, pp 389–393

30.

Maier-Hein L, Franz AM, Santos TRD, Schmidt M, Fangerau M, Meinzer HP, Fitzpatrick JM (2012) Convergent iterative closest-point algorithm to accommodate anisotropic and inhomogenous localization error. IEEE Trans Pattern Anal Mach Intell 34(8):1520–1532CrossRefPubMed

31.

Mersmann S, Seitel A, Erz M, Jähne B, Nickel F, Mieth M, Mehrabi A, Maier-Hein L (2013) Calibration of time-of-flight cameras for accurate intraoperative surface reconstruction. Med Phys 40(8):082701

32.

Müller M, Rassweiler MC, Klein J, Seitel A, Gondan M, Baumhauer M, Teber D, Rassweiler JJ, Meinzer HP, Maier-Hein L (2013) Mobile augmented reality for computer-assisted percutaneous nephrolithotomy. Int J Comput Assist Radiol Surg 8(4):663–675CrossRefPubMed

33.

Myronenko A, Song X (2010) Point set registration: coherent point drift. IEEE Trans Pattern Anal Mach Intell 32(12):2262–2275CrossRefPubMed

34.

Navab N, Traub J, Sielhorst T, Feuerstein M, Bichlmeier C (2007) Action- and workflow-driven augmented reality for computer-aided medical procedures. IEEE Comput Gr Appl 27(5):10–14CrossRef

35.

Nolden M, Zelzer S, Seitel A, Wald D, Müller M, Franz AM, Maleike D, Fangerau M, Baumhauer M, Maier-Hein L, Maier-Hein KH, Meinzer HP, Wolf I (2013) The medical imaging interaction toolkit: challenges and advances. Int J Comput Assist Radiol Surg 8(4):607–620CrossRefPubMed

36.

Qiu D, May S, Nüchter A (2009) GPU-accelerated nearest neighbor search for 3D registration. In: Proceedings of the 7th international conference on computer vision systems. Springer, pp 194–203

37.

Roberts IS, Benamore RE, Benbow EW, Lee SH, Harris JN, Jackson A, Mallett S, Patankar T, Peebles C, Roobottom C, Trail ZC (2012) Post-mortem imaging as an alternative to autopsy in the diagnosis of adult deaths: a validation study. Lancet 379(9811):136–142CrossRefPubMedCentralPubMed

38.

Rusu RB, Blodow N, Beetz M (2009) Fast point feature histograms (FPFH) for 3D registration. In: IEEE international conference on robotics and automation. IEEE, pp 3212–3217

39.

Seeling P, Fitzek FH, Ertli G, Pulipaka A, Reisslein M (2010) Video network traffic and quality comparison of VP8 and H.264 SVC. In: Proceedings of the 3rd workshop on mobile video delivery. ACM, pp 33–37

40.

Seitel A, Engel M, Sommer CM, Radeleff BA, Essert-Villard C, Baegert C, Fangerau M, Fritzsche KH, Yung K, Meinzer HP (2011) Computer-assisted trajectory planning for percutaneous needle insertions. Med Phys 38(6):3246–3259CrossRefPubMed

41.

Seitel A, Yung K, Mersmann S, Kilgus T, Groch A, dos Santos TR, Franz AM, Nolden M, Meinzer HP, Maier-Hein L (2012) MITK-ToF: range data within MITK. Int J Comput Assist Radiol Surg 7(1):87–96

42.

Sugimoto M, Yasuda H, Koda K, Suzuki M, Yamazaki M, Tezuka T, Kosugi C, Higuchi R, Watayo Y, Yagawa Y, Uemura S, Tsuchiya H, Azuma T (2010) Image overlay navigation by markerless surface registration in gastrointestinal, hepatobiliary and pancreatic surgery. J Hepato Biliary Pancreat Sci 17(5):629–636

43.

Thali MJ, Braun M, Wirth J, Vock P, Dirnhofer R (2003) 3D surface and body documentation in forensic medicine: 3-D/CAD photogrammetry merged with 3D radiological scanning. J Forensic Sci 48(6):1356–1365PubMed

44.

Thali MJ, Dirnhofer R, Vock P (2009) The virtopsy approach: 3D optical and radiological scanning and reconstruction in forensic medicine. CRC Press, Boca RatonCrossRef

45.

Urschler M, Höller J, Bornik A, Paul T, Giretzlehner M, Bischof H, Yen K, Scheurer E (2014) Intuitive presentation of clinical forensic data using anonymous and person-specific 3d reference manikins. Forensic Sci Int 241:155–166CrossRefPubMed

46.

Valette S, Chassery JM, Prost R (2008) Generic remeshing of 3D triangular meshes with metric-dependent discrete voronoi diagrams. IEEE Trans Vis Comput Gr 14(2):369–381CrossRef

47.

Yang J, Li H, Jia Y (2013) Go-ICP: solving 3D registration efficiently and globally optimally. In: International conference on computer vision. IEEE, pp 1457–1464

48.

Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334CrossRef

49.

Zhu E, Masiello I, Hadadgar A, Zary N (2014) Augmented reality in healthcare education: an integrative review. PeerJ PrePrints 2:1–22

Titel: Mobile markerless augmented reality and its application in forensic medicine
verfasst von: Thomas Kilgus
Eric Heim
Sven Haase
Sabine Prüfer
Michael Müller
Alexander Seitel
Markus Fangerau
Tamara Wiebe
Justin Iszatt
Heinz-Peter Schlemmer
Joachim Hornegger
Kathrin Yen
Lena Maier-Hein
Publikationsdatum: 01.05.2015
Verlag: Springer Berlin Heidelberg
Erschienen in: International Journal of Computer Assisted Radiology and Surgery / Ausgabe 5/2015
Print ISSN: 1861-6410
Elektronische ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-014-1106-9

Springer Professional

Abstract

Purpose

Methods

Results

Conclusions

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Weitere Artikel der Ausgabe 5/2015

Optimized viewing angles for cardiac electrophysiology ablation procedures

Subject-specific real-time respiratory liver motion compensation method for ultrasound-MRI/CT fusion imaging

Development and testing of an endoscopic pseudo-viewpoint alternating system

Automated palpation for breast tissue discrimination based on viscoelastic biomechanical properties

Neurosurgical virtual reality simulation metrics to assess psychomotor skills during brain tumor resection

Multi-slice-to-volume registration for MRI-guided transperineal prostate biopsy