Abstract
A workflow-driven cooperative working environment needs to be established in order to successfully unburden the surgeon and the OR staff from technical configuration and information-seeking tasks. An important prerequisite for autonomous situationaware adaptation of medical devices is a comprehensive representation of the operating context regarding the surgical process and situation.
We propose a hierarchical structuring of process-related and situation-related information entities and include assessment scores that intraoperative workflow information systems may provide via OR networks. The conducted experiments on the proposed assessment scores included sixty recorded brain tumour removal procedures and considered 344 distinguishable surgical situations.
A comprehensive modelling of surgical situations and process context will be a significant pre-requisite for reliable autonomous adaptation of medical devices and systems in digital operating rooms.
1 Introduction
New technologies introduced into the operating room (OR) have increased surgeons’ workload due to very time-consuming system configuration and information-seeking tasks. Although technologies for integrated operating rooms have emerged and medical devices share their information and control functions, centralized control solutions have only partly reduced surgeons’ workload. A workflow-driven cooperative working environment needs to be established in order to successfully unburden the surgeon and the OR staff [1–3].
An important prerequisite for autonomous situation-aware adaptation of medical devices is a comprehensive representation of the operating context regarding the surgical process and situation. Intelligent system’s behaviour does not solely depend on low-level tasks, but requires a more general kind of understanding of the surgical situation.
In the present work, we propose a hierarchical structuring of process-related and situation-related information entities that workflow information systems may provide online during the intervention. Additionally, we examine ways to assess and enrich the provided contextual information.
2 Material and methods
Each situation during a surgical intervention might be described from various perspectives. The relevance of an information entity highly depends on the situation and the application. For instance, the expected usage of a device or instrument might be very relevant from the resource planning point of view, but is useless for describing the current patient status. The state of the surgical process in each perspective represents a surgical situation and is an integral part of the context for any agent (medical device) which is acting in relation to that surgical process.
The hierarchical structure of contextual information describing a surgical situation at a point in time during the intervention.
2.1 Categories of process-related information
We propose a structured hierarchical representation of the process-related information as a Surgical process context. An overview of the relevant categories is depicted in Figure 1. A context instance may be generated for each point in time during the intervention. The contextual information is split into four major categories: functional, resource-related, patient-related and procedure-related.
The functional context contains the information entities associated with the functions performed to complete the surgical process. It represents the currently performed surgical work step on different granularity levels in the surgical functions subcategory including the intervention phase, the abstract surgical task and the currently performed surgical activities. The activities were represented with the commonly used perspectives: participant, action, used instrument(s) and treated anatomical structure. Additionally, parallel supporting activities, not directly related to the patient, may be represented in the Supporting functions subcategory of the functional context.
The resource-related context aggregates the information on the resources that are required for the procedure. This includes Technical resources as well as Human resources. In both categories we applied the common concept of roles. Devices, systems or individuals may be assigned to fulfil these roles accordingly.
The patient-related context is designed to reference the patient and describe the status in abstract terms related to the surgical procedure. It is not intended to include anaesthesia data. If a consumer needs these data, they should be requested from the anaesthesia workstation device possibly referenced in the technical resources.
The procedure-related context represents the information entities related to the whole intervention, for instance planned procedures (encoded as OPS-Codes) or cut-suture-times. The timestamps may also include predictions and may especially be valuable for a department-wide resource management.
2.2 Intraoperative processing
The determination of the process state may apply workflow recognition techniques; however this is not in the scope of this work. Also, the intraoperative processing may use different methods and knowledge bases, for instance statistical process models [4–6] and ontology-based representations [7, 8].
Our implementation combines model-based approaches for the generation and estimation of information entities included in the surgical process context. The intraoperative processing was based on the assumption that workflow recognition techniques provide low-level information on performed actions, used instruments and treated anatomical structures mapped onto surgical low-level tasks on a one second base (1Hz). The intervention and the abstract surgical task as well as predefined surgical events can be derived from these low-level work steps [9–11]. Briefly, the applied method was based on a network of specialized process models each describing the intervention solely in a defined granularity. Thus, the Surgical functions within the Functional context could be represented on three granularity levels. Additional process models based on Hidden Markov also allowed for qualitative statements about the current and upcoming usage of assorted medical devices (technical resources) including ultrasound devices and navigation systems [9]. Furthermore, the patient status was estimated in abstract terms related to the progress of the procedure [9, 12].
The low-level work steps were also used to estimate the remaining intervention time and the suture time [13]. These timestamps were included in the Procedure-related context. Thus the major time-dependent information entities could be provided in 1Hz based on workflow recognition input.
2.3 Online situation assessment
A Surgical process context instance was generated each second during the intervention. The surgical situation description included information entities for various perspectives onto the surgical procedure. However, there situation itself was yet not assessed in terms of relevance or significance to the course of the process.
A statistical analysis of the process instance recordings the process models are generated from may provide useful indicators to assess a given situation. We consider the following information entities of the Surgical functions: intervention phase, abstract surgical task and low-level work step as well as the abstract terms of the Patient status in the Patient-related context. A surgical process instance can be represented as a sequence of situational states. Thus, a state-transition model can be generalized from a set of annotated process instances. Each node (state) denotes a conjunction of the aforementioned information entities. The transitions and probabilities can be directly calculated from the sequential information of the considered process instances.
The proportion of process instances which contain a situation can be used as a simple indicator of how common a situation is. This measure only depends on the process instances in the training set and thus is constant during the intervention course.
In contrast to that, the probability of the transition from the previous process state to the current state expresses how expectable a situation was given the previous state. The expectability directly corresponds to the straightness of the process segment. The same situation might be very expectable to occur in a particular process phase but also be considered an exception in another section of the process.
Furthermore, we propose a characteristic score c(s) of a state si ∈ S designed to be an indicator for how characteristic a situation was for a process instance x ∈ X. The proposed calculation is based on self-information theory where
Surgical situations which occur only in a small proportion of cases but frequently within those cases get a high characteristic score. This may be interpreted as dominant or characteristic in a section of the process instance.
3 Experiments
The proposed hierarchical structuring of contextual information and the scores were integrated into our workflow information system [10, 14] to provide these comprehensive situation descriptions online via OR networks. The scoring was also included in our postoperative workflow analysis tool.
We conducted experiments with a set of recorded brain tumour removal procedures to test the meaningfulness of the proposed scores. The training set included sixty process instances recorded by a human observer on a surgical activity level. The instances were manually annotated with intervention phase, abstract surgical task and abstract patient status information. The data included 186 distinguishable low-level tasks, 4 phases, 18 abstract tasks and 16 patient status terms. The resulting state-transition model of situations consisted of 344 states. Low-level activities can be performed in different phases, or with different intentions (Abstract tasks). For instance, using the trephine to remove cranial bone material was recorded in craniotomy phase as expected but also in resection phase to extend the trepanation when the tumour tissue was already partially removed. This also illustrates the importance of contextual information to accurately describe a surgical situation instead of single a low-level activity.
Occurrence frequency and characteristic score of the surgical situations in the recorded brain tumor removal procedures.
Figure 2 depicts the occurrence frequencies and characteristic scores of situations in the recorded training set of 60 brain tumour removal procedures. The use of the CUSA device in resection phase (A) had a high characteristic score. The situation occurred in less than forty per cent of recordings, but was very dominant for the resection procedure then. In contrast, coagulation with a bipolar in the resection phase is very common in most of the process instances and thus had a higher occurrence frequency but a low characteristic score.
4 Discussion
We proposed a hierarchically structured representation of information entities related to surgical processes and situations. Such surgical process context instances are especially useful for the implementation of workflow-driven assistance functionalities and dynamic system’s behaviour in a digital operating room. Considering the surgical situations as states of the interventional suite in a state-transition model allows for the additional calculation of scores to assess the given situation.
The introduced scores might be helpful to identify particular relevant or significant situations within a population of process instances. They might also be used during the intervention to enrich the process-related information provided to medical devices and systems. Including assessment scores into the surgical situation descriptions may reduce the complexity of subsequent applications because less knowledge about the procedure needs to be included in the application itself. For instance, an assistance system for OR documentation could use the scores to identify and highlight unexpected situations that should probably be mentioned in the documentation.
A comprehensive modelling of surgical situations and process context will be a significant pre-requisite for reliable autonomous adaptation of medical devices and systems in digital operating rooms of the future. The establishment of workflow-driven technologies will require a shared terminology to preserve medical device interoperability and potentially an ontology-based extension of the hierarchical structures. This would allow the representation of inter-perspective dependencies and enable more complex assistance functionalities and automation.
Author’s Statement
Conflict of interest: Authors state no conflict of interest. Material and Methods: Informed consent: Informed consent has been obtained from all individuals included in this study. Ethical approval: The research related to human use has been complied with all the relevant national regulations, institutional policies and in accordance the tenets of the Helsinki Declaration, and has been approved by the authors’ institutional review board or equivalent committee.
References
[1] J. Sutherland and W.-J. van den Heuvel, “Towards an Intelligent Hospital Environment: Adaptive Workflow in the OR of the Future,” in Proceedings of the 39th Annual Hawaii International Conference on System Sciences, 2006. HICSS ’06, 2006, vol. 5, p. 100b–100b.10.1109/HICSS.2006.494Search in Google Scholar
[2] H. U. Lemke and M. W. Vannier, “The operating room and the need for an IT infrastructure and standards,” Int. J. Comput. Assist. Radiol. Surg., vol. 1, no. 3, pp. 117–121, Nov. 2006.10.1007/s11548-006-0051-7Search in Google Scholar
[3] K. Cleary, A. Kinsella, and S. K. Mun, “OR 2020 workshop report: Operating room of the future,” Int. Congr. Ser., vol. 1281, pp. 832–838, Mai 2005.10.1016/j.ics.2005.03.279Search in Google Scholar
[4] N. Padoy, T. Blum, S.-A. Ahmadi, H. Feussner, M.-O. Berger, and N. Navab, “Statistical modeling and recognition of surgical workflow,” Med. Image Anal., vol. 16, no. 3, pp. 632–641, Apr. 2012.10.1016/j.media.2010.10.001Search in Google Scholar PubMed
[5] T. Neumuth, P. Jannin, G. Strauss, J. Meixensberger, and O. Burgert, “Validation of Knowledge Acquisition for Surgical Process Models,” J. Am. Med. Inform. Assoc., vol. 16, no. 1, pp. 72–80, Jan. 2009.10.1197/jamia.M2748Search in Google Scholar PubMed PubMed Central
[6] J. Rosen, J. D. Brown, L. Chang, M. N. Sinanan, and B. Hannaford, “Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete Markov model,” IEEE Trans. Biomed. Eng., vol. 53, no. 3, pp. 399–413, 2006.10.1109/TBME.2005.869771Search in Google Scholar PubMed
[7] D. Katić, A.-L. Wekerle, F. Gärtner, H. Kenngott, B. P. Müller-Stich, R. Dillmann, and S. Speidel, “Ontology-based prediction of surgical events in laparoscopic surgery,” 2013, vol. 8671, p. 86711A–86711A–7.10.1117/12.2007895Search in Google Scholar
[8] D. Katić, G. Sudra, S. Speidel, G. Castrillon-Oberndorfer, G. Eggers, and R. Dillmann, “Knowledge-Based Situation Interpretation for Context-Aware Augmented Reality in Dental Implant Surgery,” in Medical Imaging and Augmented Reality, H. Liao, P. J. “Eddie” Edwards, X. Pan, Y. Fan, and G.-Z. Yang, Eds. Springer Berlin Heidelberg, 2010, pp. 531–540.10.1007/978-3-642-15699-1_56Search in Google Scholar
[9] S. Franke, J. Meixensberger, and T. Neumuth, “Multi-perspective workflow modeling for online surgical situation models,” J. Biomed. Inform., vol. 54, pp. 158–166, 2015.10.1016/j.jbi.2015.02.005Search in Google Scholar PubMed
[10] S. Franke and T. Neumuth, “A framework for event-driven surgical workflow assistance,” Biomed. Eng. Biomed. Tech., Sep. 2014.Search in Google Scholar
[11] M. Rockstroh, S. Franke, and T. Neumuth, “A workflow-driven surgical information source management,” in Int J Comput Assist Radiol Surg, Heidelberg, 2013, vol. 8 (1), pp. 189–191.Search in Google Scholar
[12] S. Franke and T. Neumuth, “A Framework for Multi-Model Surgical Workflow Management,” Biomed. Eng. Biomed. Tech., Sep. 2013.10.1515/bmt-2013-4316Search in Google Scholar PubMed
[13] S. Franke, J. Meixensberger, and T. Neumuth, “Intervention time prediction from surgical low-level tasks,” J. Biomed. Inform., vol. 46, no. 1, pp. 152–159, Feb. 2013.10.1016/j.jbi.2012.10.002Search in Google Scholar PubMed
[14] S. Franke, P. Liebmann, and T. Neumuth, “Connecting workflow management to the OR network: Design and evaluation of a bridge to enable dynamic systems behaviour,” Biomed Tech, vol. 57, p. 1, 2012.10.1515/bmt-2012-4192Search in Google Scholar PubMed
© 2015 by Walter de Gruyter GmbH, Berlin/Boston
This article is distributed under the terms of the Creative Commons Attribution Non-Commercial License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
