Computational Methods and Clinical Applications for Spine Imaging

The small butterfly shaped structure of spinal cord (SC) gray matter (GM) is challenging to image and to delineate from its surrounding white matter (WM). Segmenting GM is up to a point a trade-off between accuracy and precision. We propose a new pipeline for GM-WM magnetic resonance (MR) image acquisition and segmentation. We report superior results as compared to the ones recently reported in the SC GM segmentation challenge and show even better results using the averaged magnetization inversion recovery acquisitions (AMIRA) sequence. Scan-rescan experiments with the AMIRA sequence show high reproducibility in terms of Dice coefficient, Hausdorff distance and relative standard deviation. We use a recurrent neural network (RNN) with multi-dimensional gated recurrent units (MD-GRU) to train segmentation models on the AMIRA dataset of 855 slices. We added a generalized dice loss to the cross entropy loss that MD-GRU uses and were able to improve the results.

We describe a method to automatically predict scoliosis in Dual-energy X-ray Absorptiometry (DXA) scans. We also show that intermediate representations, which in our case are segments of body parts, help improve performance. Hence, we propose a two step process for prediction: (i) we learn to segment body parts via a segmentation Convolutional Neural Network (CNN), which we show outperforms the noisy labels it was trained on, and (ii) we predict with a classification CNN that uses as input both the raw DXA scan and also the intermediate representation, i.e. the segmented body parts. We demonstrate that this two step process can predict scoliosis with high accuracy, and can also localize the spinal curves (i.e. geometry) without additional supervision. Furthermore, we also propose a soft score of scoliosis based on the classification CNN which correlates to the severity of scoliosis.

Intra-operative ultrasound (iUS) has a considerable potential for image-guided navigation in spinal fusion surgery. Accurate registration of pre-operative computed tomography (CT) images to the iUS images is crucial for guidance. However, low image quality and bone-related artifacts in iUS render the task challenging. This paper presents a GPU-based fast CT-to-iUS rigid registration framework of a single vertebra designed for image-guided spine surgery. First, the framework involves a straightforward iUS acquisition procedure consisting in a single sweep in the cranio-caudal axis, which allows to roughly determine the initial alignment between CT and iUS images. Then, using this as a starting point, the registration is refined by aligning the gradients that are located on the posterior surface of the vertebra to obtain the final transformation. We validated our approach on a lumbosacral section of a porcine cadaver with images from T15 to L6 vertebrae. The median target registration error was 1.48 mm (\(\text {IQR} = 0.68\,\text {mm}\)), which is below the clinical acceptance threshold of 2 mm. The total registration time was \(10.79\,\text {s} \pm 1.27\,\text {s}\). We believe that our approach matches the clinical needs in terms of accuracy and computation time, which makes it a potential solution to be integrated into the surgical workflow.

We propose a method for automated grading of the vertebral endplate regions according to the Modic changes scale based on the VGG16 network architecture. We evaluate four variations of the method in a standard 9-fold cross-validation study setup on a heterogeneous dataset of 92 cases. Due to the very weak representation of the Modic Type III in the dataset, we focus on the grading of Modic Type I and Modic Type II. Despite the relatively small size of our dataset, the pipeline demonstrated a performanc1e that is similar to or better than those achieved by the state-of-the-art methods. In particular, the most performant variant achieved an accuracy of 88.0% with an average-per-class accuracy of 77.3%. When the method is used as a binary detector for the presence or not of Modic changes, the achieved average-per-class accuracy is 92.3%. Our evaluation also suggests that the so-called mixup strategy is particularly useful for this type of classification task.

Segmentation of structures in clinical images is a precursor to computer-aided detection (CAD) for many musculoskeletal pathologies. Accurate CAD systems could considerably improve the efficiency and objectivity of radiological practice by providing clinicians with image-based biomarkers calculated with minimal human input. However, such systems rarely achieve human-level performance, so extensive manual checking may be required. Their practical utility could therefore be increased by accurate error estimation, focusing manual input on the images or structures where it is needed. Standard techniques such as the minimum variance bound can estimate random errors, but provide no way to estimate any systematic errors due to model fitting failure.

We describe the use of multiple, independent sub-models to estimate both systematic and random errors. The approach is evaluated on vertebral body segmentation in lateral spinal images, demonstrating large (up to 50%) and significant improvements in the accuracy of error classification with concurrent improvements in annotation accuracy. Whilst further work is required to elucidate the definition of “independence” in this context, we conclude that the approach provides a valuable component for appearance model based CAD systems.

Accurate segmentation of intervertebral discs is a critical task in clinical diagnosis and treatment. Despite recent progress in applying deep learning to the segmentation of multiple natural image scenarios, addressing of the intervertebral disc segmentation with a small-sized training set are still challenging problems. In this paper, a new framework with fully dilated separable convolution (FDS-CNN) is proposed for the automated segmentation of the intervertebral disc using a small-sized training set. Firstly, a fully dilated separable convolutional network is designed to effectively prevent the loss of context information by reducing the number of down-sampling. Secondly, a multi-modality data fusion and augmentation strategy are proposed, which can increase the number of samples, as well as make full use of multi-modality image data. Experimental results validate the proposed framework in the MICCAI 2018 Challenge on Automatic Intervertebral Disc Localization and Segmentation from 3D Multi-modality MR Images, demonstrating excellent performance in comparison with other related segmentation methods.

The value of whole-body MRI is constantly growing and is currently employed in several bone pathologies including diagnosis and prognosis of multiple myeloma, musculoskeletal imaging and evaluation of treatment response assessment in bone metastases. Intra-patient follow-up MR images acquired over time do not only suffer from spatial misalignments caused by change in patient positioning and body composition, but also intensity inhomogeneities, making the absolute MR intensity values inherently non-comparable. The non-quantitative nature of whole-body MRI makes it difficult to derive reproducible measurement and limits the use of treatment response maps. In this work, we have investigated and compared the performance of several standardization algorithms for skeletal tissue in anatomical and diffusion-weighted whole-body MRI. The investigated method consists of two steps. First, the follow-up whole-body image is spatially registered to a baseline image using B-spline deformable registration. Secondly, an intensity standardization algorithm based on a histogram matching is applied to the follow-up image. Additionally, the use of a skeleton mask was introduced, in order to focus the accuracy of algorithms on a tissue of interest. A linear piecewise matching method using masked skeletal region showed a superior performance in comparison to the other evaluated intensity standardization methods. The proposed work helps to overcome the non-quantitative nature of whole-body MRI images, allowing for extraction of important image parameters, visualization of whole-body MR treatment response maps and assessment of severity of bone pathology based on MR intensity profile.

Scoliosis correction surgery is typically a highly invasive procedure that involves either an anterior or posterior release, which respectively entail the resection of ligaments and bone facets from the front or back of the spine, in order to make it sufficiently compliant to enable the correction of the deformity. In light of progress in other areas of surgery in minimally invasive therapies, orthopedic surgeons have begun envisioning computer simulation-assisted planning that could answer unprecedented what-if questions. This paper presents preliminary steps taken towards simulation-based surgery planning that will provide answers as to how much anterior or posterior release is truly necessary, provided we also establish the amplitude of surgical forces involved in corrective surgery. This question motivates us to pursue a medical image-based anatomical modeling pipeline that can support personalized finite elements simulation, based on models of the spine that not only feature vertebrae and inter-vertebral discs (IVDs), but also descriptive ligament models. This paper suggests a way of proceeding, based on the application of deformable multi-surface Simplex model applied to a CAD-based representation of the spine that makes explicit all spinal ligaments, along with vertebrae and IVDs. It presents a preliminary model-based segmentation study whereby Simplex meshes of CAD vertebrae are registered to the subject’s corresponding vertebrae in CT data, which then drives ligament and IVD model registration by aggregation of neighboring vertebral transformations. This framework also anticipates foreseen improvements in MR imaging that could achieve better contrasts in ligamentous tissues in the future.

In the context of the challenge of “automatic InterVertebral Disc (IVD) localization and segmentation from 3D multi-modality MR images” that took place at MICCAI 2018, we have proposed a segmentation method based on simple image processing operators. Most of these operators come from the mathematical morphology framework. Driven by some prior knowledge on IVDs (basic information about their shape and the distance between them), and on their contrast in the different modalities, we were able to segment correctly almost every IVD. The most interesting feature of our method is to rely on the morphological structure called the Three of Shapes, which is another way to represent the image contents. This structure arranges all the connected components of an image obtained by thresholding into a tree, where each node represents a particular region. Such structure is actually powerful and versatile for pattern recognition tasks in medical imaging.

Intervertebral discs are joints that lie between vertebrae in the spinal column, which absorb shock between vertebrae during activities. There is a strong correlation between lower back pain and degeneration of intervertebral discs, which may have a great impact on peoples normal life. The precise segmentation of the intervertebral disc is of great significance for the diagnosis of disc degeneration. Currently clinical practice usually manually annotates the volumetric data, which is time-consuming, tedious, needs a lot of expertise and lacks of reproducibility. In this challenge, we developed a fully automated framework that can accurately segment and locate seven intervertebral discs. First, we delicately designed a powerful segmentation network which is a 2D fully convolutional neural network with densely connected atrous spatial pyramid pooling to capture and fuse multi-scale context information. Then we used a localization network and a robust post-process scheme to distinguish different IVD instance. Further more, we proposed a novel training strategy that can make the segmentation network focus on the spine region. The effectiveness of our algorithm is proven in the challenge, we achieved the mean segmentation Dice coefficient of 90.58% and a mean localization error of 0.78 mm.

Accurate localization and segmentation of intervertebral disc (IVD) is crucial for the assessment of spine disease diagnosis. Despite the technological advances in medical imaging, IVD localization and segmentation are still manually performed, which is time-consuming and prone to errors. If, in addition, multi-modal imaging is considered, the burden imposed on disease assessments increases substantially. In this paper, we propose an architecture for IVD localization and segmentation in multi-modal magnetic resonance images (MRI), which extends the well-known UNet. Compared to single images, multi-modal data brings complementary information, contributing to better data representation and discriminative power. Our contributions are three-fold. First, how to effectively integrate and fully leverage multi-modal data remains almost unexplored. In this work, each MRI modality is processed in a different path to better exploit their unique information. Second, inspired by HyperDenseNet [11], the network is densely-connected both within each path and across different paths, granting the model the freedom to learn where and how the different modalities should be processed and combined. Third, we improved standard U-Net modules by extending inception modules [22] with two convolutional blocks with dilated convolutions of different scale, which helps handling multi-scale context. We report experiments over the data set of the public MICCAI 2018 Challenge on Automatic Intervertebral Disc Localization and Segmentation, with 13 multi-modal MRI images used for training and 3 for validation. We trained IVD-Net on an NVidia TITAN XP GPU with 16 GBs RAM, using ADAM as optimizer and a learning rate of 1\(\,\times \) 10\(^{-5}\) during 200 epochs. Training took about 5 h, and segmentation of a whole volume about 2–3 s, on average. Several baselines, with different multi-modal fusion strategies, were used to demonstrate the effectiveness of the proposed architecture.

The intervertebral discs (IVDs) segmentation and localization on medical images are important for the clinical diagnosis and research of spine diseases. In this work, we proposed a robust automatic method based on 2.5D multi-scale fully convolutional network (FCN) and geometric constraint post-processing for IVD segmentation and localization on 3D multi-modality Magnetic Resonance (MR) scans. Firstly, we designed a 2.5D multi-scale FCN. And the ensemble outputs of such three networks are used as the IVD prediction maps. The final segmentation and localization of IVDs are generated from these prediction maps with a geometric constraint post-processing method. This work ranked the first in the on-site test of MICCAI 2018 Challenge on Automatic Intervertebral Disc Localization and Segmentation from 3D Multi-modality MR Images (IVDM3Seg).

MRI is considered the gold standard in soft tissue diagnostic of the lumbar spine. Number of protocols and modalities are used – from one hand 2D sagittal, 2D angulated axial, 2D consecutive axial and 3D image types; from the other hand different sequences and contrasts are used: T1w, T2w; fat suppression, water suppression etc. Images of different modalities are not always aligned. Resolutions and field of view also vary. SNR is also different for different MRI equipment. So the goal should be to create an algorithm that covers great variety of imaging techniques.

The localization and segmentation of Intervertebral Discs (IVDs) with 3D Multi-modality MR Images are critically important for spine disease diagnosis and measurements. Manual annotation is a tedious and laborious procedure. There exist automatic IVD localization and segmentation methods on multi-modality IVD MR images, but an objective comparison of such methods is lacking. Thus we organized the following challenge: Automatic Intervertebral Disc Localization and Segmentation from 3D Multi-modality MR Images, held at the 2018 International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2018). Our challenge ensures an objective comparison by running 8 submitted methods with docker container. Experimental results show that overall the best localization method achieves a mean localization distance of 0.77 mm and the best segmentation method achieves a mean Dice of 90.64% and a mean average absolute distance of 0.60 mm, respectively. This challenge still keeps open for future submission and provides an online platform for methods comparison.