Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

01-06-2016 | Original Article

Bridging the gap between formal and experience-based knowledge for context-aware laparoscopy

Authors: Darko Katić, Jürgen Schuck, Anna-Laura Wekerle, Hannes Kenngott, Beat Peter Müller-Stich, Rüdiger Dillmann, Stefanie Speidel

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 6/2016

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Purpose

Computer assistance is increasingly common in surgery. However, the amount of information is bound to overload processing abilities of surgeons. We propose methods to recognize the current phase of a surgery for context-aware information filtering. The purpose is to select the most suitable subset of information for surgical situations which require special assistance.

Methods

We combine formal knowledge, represented by an ontology, and experience-based knowledge, represented by training samples, to recognize phases. For this purpose, we have developed two different methods. Firstly, we use formal knowledge about possible phase transitions to create a composition of random forests. Secondly, we propose a method based on cultural optimization to infer formal rules from experience to recognize phases.

Results

The proposed methods are compared with a purely formal knowledge-based approach using rules and a purely experience-based one using regular random forests. The comparative evaluation on laparoscopic pancreas resections and adrenalectomies employs a consistent set of quality criteria on clean and noisy input. The rule-based approaches proved best with noisefree data. The random forest-based ones were more robust in the presence of noise.

Conclusion

Formal and experience-based knowledge can be successfully combined for robust phase recognition.

previous article Single-view X-ray depth recovery: toward a novel concept for image-guided interventions

next article Guided ultrasound calibration: where, how, and how many calibration fiducials

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Skagestad P (1993) Thinking with machines: intelligence augmentation, evolutionary epistemology, and semiotic. J Soc Evol Syst 16(2):157–180CrossRef

Woods DD, Patterson ES, Roth EM (2002) Can we ever escape from data overload? A cognitive systems diagnosis. Cogn Technol Work 4(1):22–36CrossRef

Dixon B, Daly MJ, Chan H, Vescan AD, Witterick IJ, Irish JC (2013) Surgeons blinded by enhanced navigation: the effect of augmented reality on attention. Surg Endosc 27(2):454–461CrossRefPubMed

Joyce JP, Lapinsky GW (1983) A history and overview of the safety parameter display system concept. IEEE Trans Nucl Sci 30(1):744–749CrossRef

Kranzfelder M, Staub C, Fiolka A, Schneider A, Gillen S, Wilhelm D, Friess H, Knoll A, Feussner H (2013) Toward increased autonomy in the surgical OR: needs, requests, and expectations. Surg Endosc 27(5):1681–1688CrossRefPubMed

Linte CA, Davenport KP, Cleary K, Peters C, Vosburgh KG, Navab N, Edwards PE, Jannin P, Peters TM, Holmes DR, Robb RA (2013) On mixed reality environments for minimally invasive therapy guidance: systems architecture, successes and challenges in their implementation from laboratory to clinic. Comp Med Imaging Graph 37(2):83–97CrossRef

Lalys F, Jannin P (2014) Surgical process modelling: a review. Int J Comput Assist Radiol Surg 9(3):495–511. doi:10.1007/s11548-013-0940-5

Blum T, Feussner H, Navab N (2010) Modeling and segmentation of surgical workflow from laparoscopic video. In: Jiang T, Navab N, Pluim JPW, Viergever MA (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2010, vol 6363. Springer, Berlin, pp 400–407

Suzuki T, Sakurai Y, Yoshimitsu K, Nambu K, Muragaki Y, Iseki H (2010) Intraoperative multichannel audio–visual information recording and automatic surgical phase and incident detection. In: IEEE EMBS, pp 1190–1193

10.

Blum T, Padoy N, Feussner H, Navab N (2008) Workflow mining for visualization and analysis of surgeries. Int J Comput Assist Radiol Surg 3(5):379–386CrossRef

11.

Bouarfa L, Jonker PP, Dankelman J (2010) Discovery of high-level tasks in the operating room. J Biomed Inform 44(3):455–462CrossRefPubMed

12.

Ahmadi A, Sielhorst T, Stauder R, Horn M, Feussner H, Navab N (2007) Recovery of surgical workflow without explicit models. In: Proceedings of the MICCAI, pp 420–428

13.

Stauder R, Okur A, Peter L, Schneider A, Kranzfelder M, Feussner H, Navab N (2014) Random forests for phase detection in surgical workflow analysis. In: Proceedings of the IPCAI

14.

Reiley CE, Lin HC, Varadarajan B, Vagvolgyi B, Khudanpur S, Yuh DD, Hager GD (2008) Automatic recognition of surgical motions using statistical modeling for capturing variability. Stud Health Technol Inform 132:396–401

15.

Lalys F, Riffaud L, Morandi X, Jannin P (2011) Surgical phases detection from microscope videos by combining SVM and HMM medical computer vision. Recognition techniques and applications in medical imaging

16.

Padoy N, Blum T, Ahmadi SA, Feussner H, Berger MO, Navab N (2010) Statistical modeling and recognition of surgical workflow. Med Image Anal 16(3):632–641CrossRefPubMed

17.

Neumuth T, Jannin P, Schlomberg J, Meixensberger J, Wiedemann P, Burgert O (2011) Analysis of surgical intervention populations using generic surgical process models. Int J Comput Assist Radiol Surg 6(1):59–71CrossRefPubMedPubMedCentral

18.

Burgert O, Neumuth T, Lempp F, Mudunuri R, Meixensberger J, Strauss G, Dietz A, Jannin P, Lemke HU (2006) Linking top-level ontologies and surgical workflows. Int J Comput Assist Radiol Surg 1(1):437–438

19.

Jannin P, Morandi X (2007) Surgical models for computer-assisted neurosurgery. Neuroimage 37(3):783–791CrossRefPubMed

20.

Neumuth T, Kaschek B, Neumuth D, Ceschia M, Meixensberger J, Strauss G, Burgert O (2010) An observation support system with an adaptive ontology-driven user interface for the modeling of complex behaviors during surgical interventions. Behav Res Methods 42(4):1049–1058CrossRefPubMed

21.

Neumuth D, Loebe F, Herre H, Neumuth T (2011) Modeling surgical processes: a four-level translational approach. Artif Intell Med 51(3):147–161CrossRefPubMed

22.

Lalys F, Bouget B, Riffaud R, Jannin P (2013) Automatic knowledge-based recognition of low-level tasks in ophthalmological procedures. Int J CARS 8:39–49CrossRef

23.

Katic D, Julliard C, Wekerle AL, Kenngott H, Müller-Stich BP, Dillmann R, Speidel S, Jannin P, Gibaud B (2015) LapOntoSPM: an ontology for laparoscopic surgeries and its application to surgical phase recognition. Int J CARS 10:1427–1434CrossRef

24.

Katic D, Wekerle AL, Grtner F, Kenngott H, Müller-Stich BP, Dillmann R, Speidel S (2014) Knowledge-driven formalization of laparoscopic surgeries for rule-based intraoperative context-aware assistance. In: IPCAI

25.

Breiman L (2001) Random forests. Mach Learn 45(1):5–32CrossRef

26.

Katic D, Wekerle AL, Gaertner F, Kenngott H, Müller-Stich BP, Dillmann E, Speidel S (2013) Ontology-based prediction of surgical events in laparoscopic surgery. In: Proceedings of the SPIE

27.

Forestier G, Riffaud L, Jannin P (2015) Automatic phase prediction from low-level surgical activities. Int J Comput Assist Radiol Surg 20(6):833–884CrossRef

28.

Horrocks I, Patel-Schneider PF, Boley H, Tabet S, Grosof B, Dean M (2004) SWRL: a semantic web rule language combining OWL and RuleML. W3C Member Submission. https://www.w3.org/Submission/SWRL/. Accessed 25 Mar 2016

29.

Ma H, Wang y (2009) Cultural algorithm based on particle swarm optimization for function optimization. In: Fifth international conference on natural computation

30.

Kennedy J, Eberhart R (1995) Particle swarm optimization. In: IEEE international conference on NN

31.

Kobti Z, Reynolds R, Kohler T (2004) Agent-based modeling of cultural change in swarm using cultural algorithms. In: SWARMFEST

32.

Neumuth T, Jannin P, Strauss G, Meixensberger J, Burgert O (2009) Validation of knowledge acquisition for surgical process models. J Am Med Inform Assoc 16(1):72–80

33.

Speidel S, Benzko J, Sudra G, Azad P, Müller-Stich BP, Gutt , Dillmann R (2009) Automatic classification of minimally invasive instruments based on endoscopic image sequences. In: SPIE medical imaging

34.

Yang Y, Calmet J (2005) OntoBayes: an ontology-driven uncertainty model. In: International conference on intelligent agents, web technologies and internet commerce

Title: Bridging the gap between formal and experience-based knowledge for context-aware laparoscopy
Authors: Darko Katić
Jürgen Schuck
Anna-Laura Wekerle
Hannes Kenngott
Beat Peter Müller-Stich
Rüdiger Dillmann
Stefanie Speidel
Publication date: 01-06-2016
Publisher: Springer Berlin Heidelberg
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 6/2016
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-016-1379-2

Springer Professional

Abstract

Purpose

Methods

Results

Conclusion

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Other articles of this Issue 6/2016

Device- and system-independent personal touchless user interface for operating rooms

In vivo validation of spatio-temporal liver motion prediction from motion tracked on MR thermometry images

Calibration of RGBD camera and cone-beam CT for 3D intra-operative mixed reality visualization

On-demand calibration and evaluation for electromagnetically tracked laparoscope in augmented reality visualization

Guided ultrasound calibration: where, how, and how many calibration fiducials

Detection of prostate cancer using temporal sequences of ultrasound data: a large clinical feasibility study

Premium Partner