Surgery Simulation and Soft Tissue Modeling

This paper outlines the limitations of the rigid body assumption in image guided interventions and describes how intra-operative imaging provides a rich source of information on spatial location of key structures allowing a pre-operative plan to be updated during an intervention. Soft tissue deformation and variation from an atlas to a particular individual can both be determined using non-rigid registration. Classic methods using free-form deformations have a very large number of degrees of freedom. Three examples - motion models, biomechanical models and statistical shape models - are used to illustrate how prior information can be used to restrict the number of degrees of freedom of the registration algorithm to produce solutions that could plausibly be used to guide interventions. We provide preliminary results from applications in each.

The context of this research is the development of a pedagogical surgery simulator for colon cancer removal. More precisely, we would like to simulate the gesture which consists of moving the small intestine folds away from the cancerous tissues of the colon. This paper presents a method for animating the small intestine and the mesentery (the tissue that connects it to the main vessels) in real-time, thus enabling user-interaction through virtual surgical tools during the simulation. The main issue that we solve here is the real-time processing of multiple collisions and self-collisions that occur between the intestine and mesentery folds.

When maxillofacial surgery is proposed as a treatment for a patient, the type of osteotomy and its influence on the facial contour is of major interest. To design the optimal surgical plan, 3D image-based planning can be used. However, prediction of soft tissue deformation due to skeletal changes, is rather complex. The soft tissue model needs to incorporate the characteristics of living tissues.Since surgeon and patient are interested in the expected facial contour some months after surgery when swelling has disappeared, features specific to living tissues need to be modelled. This paper focusses on modelling of tissue growth using finite element methods. This growth is induced by stress resulting from the surgical procedure. We explain why modelling growth is needed and propose a model. We apply this model to 4 patients treated with unilateral mandibular distraction and compare these soft tissue predictions with the postoperative CT image data.

In this paper, we present a system dedicated to the simulation of various physically-based and mainly deformable objects. Its main purpose is surgical simulation where many models are necessary to simulate the organs and the user’s tools. In our system, we found convenient to decompose each simulated model in three units: The mechanical, the visual and the collision units. In practice, only the third unit is actually constrained, since we want to process collisions in a unified way. We choose to rely on a fast penalty-based method which uses approximation of the objects depth map by spheres. The simulation is sufficiently fast to control force feedback devices.

This paper introduces the Radial Elements Method — REM for the simulation of deformable objects. The REM was conceived for the real time, dynamic simulation of deformable objects. The method uses a combination of static and dynamic approaches to simulate deformations and dynamics of highly deformable objects. The real time performance of the method and its intrinsic properties of volume conservation, modeling based in material properties and simple meshing make it particularly attractive for soft tissue modeling and surgery simulation.

The construction of surgery simulators will be a key tool in the development and diffusion of minimally invasive surgery. Nowadays, most simulators are oriented to training surgeons in only one surgery technique. Most of them only permit the modelling of tissues with only one kind of deformable model. In this paper, we present our generic surgery simulator for minimally invasive surgery. In this surgery simulator, surgeons can construct any surgery scenario that they want to practice. Our surgery simulator permits the surgeons to select the deformable model that best adjusts to the biomechanical properties of each organ. Once the surgeon has finished the training, our surgery simulator can generate a report that contains an assessment evaluation of that training.

Towards semi-automated transfer of preoperatively planned bone repositionings in the operation theatre, this paper presents a solution by performing preoperative bending of common titanium plate implants. The approach is based on our simulation system KasOp which defines preoperatively bone cut trajectories and bone repositionings. According to physiological matters the segments are repositioned and the computer models of the implants are bended in respect to physical constraints following the physiological shape of a bone of same age and sex. We are in process of developing a bending device which can bend standard titanium implants corresponding to the simulation. Therefore, precast implants can be used during surgery speeding up the fixation procedure of the cut bone segments.

Realistic mechanical models of biological soft tissues are a key issue to allow the implementation of reliable systems to aid on orthopedic diagnosis and surgery planning. We are working to develop a computerized soft tissues model for bio-tissues based on a mass-spring-like approach. In this work we present several experiments towards the parameterization of our model from the elastic properties of real materials.

We present a novel method for tracking the motion of the myocardium in tagged magnetic resonance (MR) images of the heart using a nonrigid registration algorithm based on a cylindrical free-form deformation (FFD) model and the optimization of a cost function based on normalized mutual information (NMI). The new aspect of our work is that we use a FFD de.ned in a cylindrical rather than a Cartesian coordinate system. This models more closely the geometry and motion of the left ventricle (LV). Validation results using a cardiac motion simulator and tagged MR data from 6 normal volunteers are also presented.

In this paper we present the current state of our research on simulation of temporal bone surgical procedures. We describe the results of tests performed on a virtual surgical training system for middle ear surgery. The work is aimed to demonstrate how expert surgeons and trainees can effectively use the system for training and assessment purposes. Preliminary kinematic and dynamic analysis of simulated mastoidectomy sessions are presented. The simulation system used is characterized by a haptic component exploiting a bone-burr contact and erosion simulation model, a direct volume rendering module as well as a time-critical particle system to simulate secondary visual effects, such as bone debris accumulation, blooding, irrigation, and suction.

This paper presents the haptic interaction method when the interaction occurs at several points simultaneously. In many virtual training systems that interact with a virtual object, the haptic interface is modeled as a point. However, in the real world, the portion interacting with real material is not a point but rather multiple points, i.e., an area. In this paper, we address an area-based haptic rendering technique that enables the user to distinguish hard regions from softer ones by providing the distributed reflected force and the sensation of rotation at the boundary. We have used a shape retaining chain linked model that is suitable for real-time applications in order to develop a fast area-based volume haptic rendering method for volumetric objects. We experimented with homogeneous and non-homogeneous virtual objects consisting of 421,875 (75×75×75) volume elements.

In this paper, we present a 3D reconstruction approach of a liver tumour model from a sequence of 2D MR parallel cross-sections, and the integration of this reconstructed 3D model with a mechanical tissue model. The reconstruction algorithm uses shape-based interpolation and extrapolation. While interpolation generates intermediate slices between every pair of adjacent input slices, extrapolation performs a smooth closing of the external surface of the model. Interpolation uses morphological morphing, while extrapolation is based on smoothness surface constraints. Local surface irregularities are further smoothed with Taubin’s surface fairing algorithm [5]. Since tumour models are to be used in a planning and simulation system of image-guided cryosurgery, a mechanical model based on a non-linear tensor-mass algorithm was integrated with the tumour geometry. Integration allows the computation of fast deformations and force feedback in the process of cryoprobe insertion.

In several medical applications it is necessary to have a good reconstruction of approximately tubular structures — mainly blood vessels but also intestine or bones — providing a description of both the internal lumen (usually a triangulated surface) and its networked structure (skeleton). This description should be such that it allows lengths and diameters estimation. Several methods have been proposed for these tasks, each one with advantages and drawbacks and, typically, specialized to a particular application. We focused our attention on methods making as few assumptions as possible on the structure to be determined in order to capture also anomalous features like bulges and bifurcations. We looked for a method able to obtain surfaces that are smooth, with a limited number of triangles but accurate and skeletons that are continuously connected and centered. The results of our work is the use of customized deformable surface and multi-scale regularized voxel coding centerlines to obtain geometries and skeletons with the desired properties. The algorithms are being tested for real clinical analysis and results are promising.

Maxillofacial surgery treats abnormalities of the skeleton of the head. Skull remodelling implies osteotomies, bone fragment repositioning, restoration of bone defects, inserting implants, . . . . Recently, the use of 3D image-based surgery planning systems is more and more accepted in this field. Although the bone-related planning concepts and methods are maturing, prediction of soft tissue deformation needs further fundamental research. In this paper we present a tetrahedral soft tissue model that can be used in a surgery planning system to predict soft tissue changes due to skeletal changes. Our model consists of mass points connected by springs. We propose a way to directly calculate the deformation of the model due to external changes. To achieve fast calculations we take advantage of the fact that most deformations are local and we compare our results with pre-computed reference models, to prove the accuracy of our model.

Surgery simulation is one of the largest applications of Medical Virtual Reality. In order to perform the real-time simulation, we constructed an elastic organ model known as a sphere-filled model. This proposed organ model allows us to perform surgical maneuvers such as pushing, pinching and incising and show the deformation of the inner structures such as blood vessels on our system. In addition, we tried to obtain haptic sensation with the patients’ organs in a surgical simulation. Developed system made it possible to handle elastic organs with two force feedback devices attached to the both of users’ hands. At the same time, we have been developing a VR cockpit suited for virtual surgery and tele-surgery. Using our VR cockpit, our system allows us to provide the users with an environment closely resembling the open surgery situation.

Methods and fundamental considerations for adding force feedback to a surgery simulator are presented. As an example, the virtual endoscopic surgery trainer “VS-One”, developed at the Forschungszentrum Karlsruhe, is taken. An overview on building a general interface to force feedback applications is given. Implementations and satellite modules of the simulation software KISMET are presented as the results. The concept and the implementations are found to be flexible, stable and for universal use. Conclusions are drawn from the results regarding actual and further developments.

This paper presents a methodology to simulate 3D cuts in deformable objects. It uses an explicit finite element approach to simulate deformations in real-time. We use a non-linear strain tensor formulation (Green tensor) to allow large displacements. Haptics is used to allow touching sensation of the cutting procedure.

Breast cancer and other breast pathologies often manifest as an area of increased tissue stiffness. The gold standard for breast cancer screening is mammography, which records the radioopacity of tissues in the breast, and therefore depends on factors other than stiffness. Tactile imaging uses an array of pressure sensors to noninvasively record the palpable extent of breast tissue stiffness. Tactile imaging quantifies palpation, and holds promise for increasing the positive predictive value of screening mammography by highlighting areas of abnormal stiffness. We propose a method for registering tactile images obtained in the same plane and immediately after the corresponding mammogram. A finite element model-based approach is presented which is used to account for the spreading of the breast tissue induced by the mammographic compression that is not present in obtaining the tactile image. We devise an algorithm that can register the modeled tactile images to within 6% of the modeled mammograms for breasts of varying size and stiffness. Clinical mammograms and tactile images were collected on 11 subjects, and the registration algorithm applied to the images. The registered tactile images and mammograms correlate well over stiff, radioopaque areas such as glandular and fibrous tissue, and highlight areas of increased stiffness not indicated by the mammogram alone.

Realistic haptic interaction in volume sculpting is a decisive prerequisite for successful simulation of bone surgery.We present a haptic rendering algorithm, based on a multi-point collision detection approach which provides realistic tool interactions. Both haptics and graphics are rendered at sub-voxel resolution, which leads to a high level of detail and enables the exploration of the models at any scale. With a simulated drill bony structures can be removed interactively. The characteristics of the real drilling procedure like material distribution around the drill are considered to enable a realistic sensation. All forces are calculated at an extra high update rate of 6000 Hz which enables rendering of drilling vibrations and stiff surfaces. As a main application, a simulator for petrous bone surgery was developed. With the simulated drill, access paths to the middle ear can be studied. This allows a realistic training without the need for cadaveric material.

A critical challenge for the neurosurgeon during surgery is to be able to preserve healthy tissue and minimize the disruption of critical anatomical structures while at the same time removing as much tumor tissue as possible. Over the past several years we have developed intraoperative image processing algorithms with the goal of augmenting the surgeon’s capacity to achieve maximal tumor resection while minimizing the disruption to normal tissue. The brain of the patient often changes shape in a nonrigid fashion over the course of a surgery, due to loss of cerebrospinal fluid, concomitant pressure changes, the impact of anaesthetics and the surgical resection itself. This further increases the challenge of visualizing and navigating critical brain structures. The primary concept of our approach is to exploit intraoperative image acquisition to directly visualize the morphology of brain as it changes over the course of the surgery, and to enhance the surgeon’s capacity to visualize critical structures by projecting extensive preoperative data into the intraoperative configuration of the patient’s brain.Our approach to tracking brain changes during neurosurgery has been previously described. We identify key structures in volumetric preoperative and intraoperative scans, and use the constraints provided by the matching of these key surfaces to compute a biomechanical simulation of the volumetric brain deformation. The recovered volumetric deformation field can then be applied to preoperative data sets, such as functional MRI (fMRI) or diffusion tensor MRI (DT-MRI) in order to warp this data into the new configuration of the patient’s brain. In recent work we have constructed visualizations of preoperative fMRI and DT-MRI, and intraoperative MRI showing a close correspondence between the matched data. p ]A further challenge of intraoperative image processing is that augmented visualizations must be presented to the neurosurgeon at a rate compatible with surgical decision making. We have previously demonstrated our biomechanical simulation of brain deformation can be executed entirely during neurosurgery. We used a generic atlas to provide surrogate information regarding the expected location of critical anatomical structures, and were able to project this data to match the patient and to display the matched data to the neurosurgeon during the surgical procedure. The use of patient-specific DTI and fMRI preoperative data significantly improves the localization of critical structures. The augmented visualization of intraoperative data with relevant preoperative data can significantly enhance the information available to the neurosurgeon.

Mechanical properties of the myocardium have been investigated intensively in the last four decades. Due to the nonlinearity and history dependence of the myocardial deformation, many complex strain energy functions have been used to describe the stress-strain relationship of myocardium. These functions are good at fitting in-vitro experimental data from myocardial stretch testing. However it is difficult to model in-vivo myocardium by using the strain energy functions. In a previous paper [24], we have implemented transversely anisotropic material model to estimate in-vivo strain-stress analysis in the myocardium. In this work, the fiber orientation is updated at each time step from the end of diastole to the end of systole, and the stiffness matrix is recalculated using the current fiber orientation. We also extended our model to include residual ventricular stresses and time dependent blood pressure in the left ventricle cavity.

This article describes an experimental protocol to obtain in vivo macroscopic measures of the cardiac electrical activity in a canine heart coupled with simulations done using macroscopic models of the canine myocardium. Electrical propagation simulations are conducted along with preliminary qualitative comparisons. Two different models are compared, one built from dissection and highly smoothed and one measured from Diffusion Tensor Imaging (DTI). Validating a macroscopic model with in vivo measurements of the electrical activity should allow a future use of the model in a predictive way, for instance in radiofrequency ablation planning.

We present a system to assist in the treatment of tachycardia patients by catheter ablation.In an augmented reality framework we combine a patient specific preoperative MR model, constructed from a set of transverse, coronal and sagittal images, with intra-cardial voltage potential measurements and fluoroscopic imaging to guide the electrophysiologist. The registration of the model and the fluoroscopic images, which is done by a visual matching technique, enables an easy transfer of the measurement to the pre-operative model. By visualizing annotations of different tissue types and of the measurements, extra insight is gathered about the problem, resulting in improved patient care.Because of its low cost and similar advantages we believe our approach can compete with existing commercial solutions, which rely on dedicated hardware and costly catheters. First clinical evaluation on 31 patients indicate a considerable advantage in the diagnosis and treatment.Our future work will consist of improving 2D-3D registration and further automating the measurement procedure.

Modeling of cardiac electro-mechanics enables and simplifies understanding of physiology and pathophysiology of the heart. In this work a model is presented, which allows the reconstruction of macroscopic electro-mechanical processes in the left ventricle of small mammals. The model combines a three-dimensional model of left ventricular anatomy represented as truncated ellipsoid with an integrated electromechanical model. The integrated model includes electrophysiological, force development and elastomechanical models of myocardium. The model is illustrated by simulations, which reflect the behavior of an extracorporated heart. These simulations yield temporal distributions of electrophysiological parameters as well as descriptions of electrical propagation and mechanical deformation. The simulations show the connection between cellular electrophysiology, electrical excitation propagation, force development, and mechanical deformation.

Sleep Apnea Syndrome (SAS) is defined as a partial or total closure of the patient upper airways during sleep. The term “collapsus” (or collapse) is used to describe this closure. From a fluid mechanical point of view, this collapse can be understood as a spectacular example of fluid-walls interaction. Indeed, the upper airways are delimited in their largest part by soft tissues having different geometrical and mechanical properties: velum, tongue and pharyngeal walls. Airway closure during SAS comes from the interaction between these soft tissues and the inspiratory flow. The aim of this work is to understand the physical phenomena at the origin of the collapsus and the metamorphosis in inspiratory flow pattern that has been reported during SAS. Indeed, a full comprehension of the physical conditions allowing this phenomenon is a prerequisite to be able to help in the planning of the surgical gesture that can be prescribed for the patients. The work presented here focuses on a simple model of fluid-walls interactions. The equations governing the airflow inside a constriction are coupled with a Finite Element biomechanical model of the velum. The geometries of this model is extracted from a single midsagittal radiography of a patient. The velar deformations induced by airflow interactions are computed, presented, discussed and compared to measurements collected onto an experimental setup.

Our purpose in this article is to superimpose a 3D model of the liver, its vessels and tumors (reconstructed from CT images) on external video images of the patient for hepatic surgery guidance. The main constraints are the robustness, the accuracy and the computation time. Because of the absence of visible anatomical landmarks and of the “ cylindrical” shape of the upper abdomen, we used some radio-opaque fiducials. The classical least-squares method assuming that there is no noise on the 3D point positions, we designed a new Maximum Likelihood approach to account for this existing noise and we show that it generalizes the classical approaches. Experiments on synthetic data provide evidences that our new criterion is up to 20% more accurate and much more robust, while keeping a computation time compatible with realtime at 20 to 40 Hz. Eventually, careful validation experiments on real data show that an accuracy of 2 mm can be achieved within the liver.

There is a need to determine biomechanical properties of liver tissue to develop realistic elastic deformable liver model for computer aided surgery. In this report, we introduced a method to measure mechanical properties using surgical instant adhesive (surgical glue). The method made easier to define the mechanical boundary conditions for test pieces. It also makes it possible to conduct both compression and elongation test on the same test piece. In actual deformation of liver during surgical intervention, the tissue is subject both to compression and elongation. Identification of mechanical properties in the range where mechanical force changes from compression to elongation is important. We can identify the stress-strain relationship of liver samples in the transition range from compression to elongation. We also investigated viscoelastic properties by compressing the sample at different velocities. The obtained results can be applied to non linear FEM analysis of liver tissue.

Soft tissue artefact is the most invalidating source of error in human motion analysis. This error is caused by the erroneous assumption that markers on the skin surface are rigidly connected to the underlying bone. Several methods have been proposed in the literature to compensate for this spurious effect by mathematical modelling of the interposed soft tissues. Validation of these methods has been performed only on simulated data or on a small sample of data acquired from subjects mounting devices which limit skin motion. In the present study, the performance of one of the most recent compensation methods was evaluated using experimental data acquired combining stereophotogrammetry and 3D video-fluoroscopy. The effectiveness of the compensation method was found strongly dependent on the modelling form assumed for the motion of the markers in the bone frame. Even when the compensation produced a significant advantage in the evaluation of bone orientation, the estimation of position was critical.

In this paper we introduce the extendible and cross-platform software framework Julius. Julius combines both pre-operative planning and intraoperative assistance within one single environment. In this paper we discuss three aspects of Julius: the medical data processing, the visualization pipeline and the interaction. Each aspect provides interfaces that allow to extend the application with own algorithms and to build complex applications. We believe that this approach facilitates the development of image guided navigation and simulation procedures for computer-aided-surgery.

This paper presents a method to artificially re-create haptic feedback while moving and sliding an arbitrary virtual tool against a virtual deformable body with nonlinear elastic properties. The computation of the response in such general cases is a task which does not yet admit computational solutions suitable for realtime implementation. To address this, we describe an approach based on the bookkeeping of forcede deflections curves stored at the nodes of a triangulated body surface. For realism, normal and lateral deformations at each node are represented in a range of deflection distances. The response everywhere is synthesized via area interpolation of response curves stored at the nodes of the mesh. The mathematical continuity of the synthetic response is the result of both local coordinates interpolation and of response function interpolation, which previous methods did not account for. This guarantees the absence of haptic‘ clicks’ and ‘pops’ which are unacceptable artifacts in high fidelity simulations. Sliding contacts are also considered.

Stereotactic neurosurgery for Parkinson’s disease consists of stimulating deep nuclei of the brain. Although target coordinates are calculated with high precision on the pre-operative images, cerebrospinal fluid (CSF) leakage during the procedure can lead to a brain deformation and cause potential error with respect to the surgical planning. In this paper, we propose a patient-specific biomechanical model of the brain able to recover the global deformation of the brain during this type of neurosurgical procedure. Such a model could be used to update the pre-operative planning and balance the mechanical effects of the intra-operative brain shift.

The deformation of myocard takes a vital part in the pumping function of the heart muscle. Knowledge of myocard anatomy and physiology makes it possible to create models of cardiac behavior. These models can be used for surgery planning, educational and research purposes. Simulations can be performed, which are beyond the capability of physical experiments. Volume models of myocard are necessary for realistic simulations. The deformation model presented in this work is based on a mass spring system parameterized by a continuum mechanics deformation model. Electrophysiology and excitation propagation of myocard were simulated and the resulting force development was used as input for the deformation models. Electromechanical coupled simulations with simple myocard geometries were carried out to show the capability of the deformation model in reconstruction of cardiac deformation.

This paper presents ongoing research on a semi-automatic method for computing, from CT and MR data, patient-specific anatomical models used in surgical simulation. Surgical simulation is a software implementation enabling a user to interact, through virtual surgical tools, with an anatomical model representative of relevant tissues and endowed with realistic constitutive properties. Up to now, surgical simulators have generally been characterized by their reliance on a generic anatomical model, typically obtained at the cost of extensive user interaction, and by biomechanical computations based on mass-spring networks.We propose a minimally supervised procedure for extracting from a set of CT and MR scans a highly descriptive tissue classification, a set of triangulated surfaces coinciding with relevant tissue boundaries, and volumetric meshes bounded by these surfaces and comprised of tetrahedral elements of homogeneous tissue. In this manner, a series of models could be obtained with little user interaction, allowing surgeons to be trained on a large set of pathologies which are clinically representative of those they are likely to encounter. The application of this procedure to the simulation of pituitary surgery is described. Furthermore, the resolution of the surface and tissue meshes is explicitly controllable with a few simple parameters. In turn, the target mesh resolution can be expressed as a radially varying function from a central point, in this case coinciding with a point on the pituitary gland.A further objective is to produce anatomical models which can interact with a published finite element-based biomechanical simulation technique which partitions the volume into separate parent and child meshes: the former sparse and linearly elastic; the latter dense, centered on the region of clinical interest and possibly nonlinearly elastic.

Periodic deformations of organs which are due to respiratory movements may be critical disturbances for surgeons manipulating robotic control systems during laparoscopic interventions or tele-surgery. Indeed, the surgeon has to manually compensate for these motions if accurate gestures are needed, like, e.g., during suturing. This work presents a model predictive control scheme that is applied to the problem of maintaining a constant distance in the endoscopic images from a surgical tool’s tip to the organ’s surface. A new optimization criterion is developed for an unconstrained generalized predictive controller based on a repetitive input-output model, where contributions of the control input to reference tracking and to disturbance rejection are split and computed separately. Thanks to this approach, mechanical filtering of the repetitive disturbances and teleoperation by the surgeon can run simultaneously and independently on the robot arm. The system is tested on both an endotraining box with a surgical robot and in in vivo conditions on a living pig. Results are shown to validate the control scheme and its application.

In the last few years, radiofrequency ablation has become one of the most promising techniques to treat liver tumors. But radiologists have to face the difficulty of planning their treatment while only relying on 2D slices. We present here a realistic radiofrequency ablation simulation tool, coupled with a 3D reconstruction and visualization project. They help radiologists to have a better visualization of patients anatomic structures and pathologies, and allow them to easily find an adequate treatment.

Realistic generation of variable anatomical organ models and pathologies are crucial for a sophisticated surgical training simulator. A training scene needs to be different in every session in order to exhaust the full potential of virtual reality based training. We previously reported on a cellular automaton able to generate leiomyomas found in the uterine cavity. This paper presents an alternative approach for the design of macroscopic findings of pathologies and describes the incorporation of these models into a healthy virtual organ. The pathologies implemented are leiomyomas and polyps protruding to different extents into the uterine cavity. The results presented are part of a virtual reality based hysteroscopy simulator that is under development.