Fetal, Infant and Ophthalmic Medical Image Analysis

Accurate estimation of intracranial volume (ICV) is key in neuro-imaging-based volumetric studies, since estimation errors directly propagate to the ICV-corrected volumes used in subsequent analyses. ICV estimation through registration to a reference atlas has the advantage of not requiring manually delineated data, and can thus be applied to populations for which labeled data might be inexistent or scarce, e.g., preterm born animal models. However, such method is not robust, since the estimation depends on a single registration. Here we present a groupwise, template-free ICV estimation method that overcomes this limitation. The method quickly aligns pairs of images using linear registration at low resolution, and then computes the most likely ICV values using a Bayesian framework. The algorithm is robust against single registration errors, which are corrected by registrations to other subjects. The algorithm was evaluated on a pilot dataset of rabbit brain MRI (\(N=7\)), in which the estimated ICV was highly correlated (\(\rho =0.99\)) with ground truth values derived from manual delineations. Additional regression and discrimination experiments with human hippocampal volume on a subset of ADNI (\(N=150\)) yielded reduced sample sizes and increased classification accuracy, compared with using a reference atlas.

As maturation of neural networks continues throughout childhood, brain lesions insulting immature networks have different impact on brain function than lesions obtained after full network maturation. Thus, longitudinal studies and analysis of spatial and temporal brain signal correlations are a key component to get a deeper understanding of individual maturation processes, their interaction and their link to cognition. Here, we assess the connectivity pattern deviation of developing resting state networks after ischaemic stroke of children between 7 and 17 years. We propose a method to derive a reorganisational score to detect target regions for overtaking affected functional regions within a stroke location. The evaluation is performed using rs-fMRI data of 16 control subjects and 16 stroke patients. The developing functional connectivity affected by ischaemic stroke exhibits significant differences to the control cohort. This suggests an influence of stroke location and developmental stage on regenerating processes and the reorganisational patterns.

Ultrasound is the primary imaging method for prenatal screening and diagnosis of fetal anomalies. Thanks to its non-invasive and non-ionizing properties, ultrasound allows quick, safe and detailed evaluation of the unborn baby, including the estimation of the gestational age, brain and cranium development. However, the accuracy of traditional 2D fetal biometrics is dependent on operator expertise and subjectivity in 2D plane finding and manual marking. 3D ultrasound has the potential to reduce the operator dependence. In this paper, we propose a new random forest-based segmentation framework for fetal 3D ultrasound volumes, able to efficiently integrate semantic and structural information in the classification process. We introduce a new semantic features space able to encode spatial context via generalized geodesic distance transform. Unlike alternative auto-context approaches, this new set of features is efficiently integrated into the same forest using contextual trees. Finally, we use a new structured labels space as alternative to the traditional atomic class labels, able to capture morphological variability of the target organ. Here, we show the potential of this new general framework segmenting the skull in 3D fetal ultrasound volumes, significantly outperforming alternative random forest-based approaches.

We propose a fast feature-based rigid registration framework with a novel feature saliency detection technique. The method works by automatically classifying candidate image points as salient or non-salient using a support vector machine trained on points which have previously driven successful registrations. Resulting candidate salient points are used for symmetric matching based on local descriptor similarity and followed by RANSAC outlier rejection to obtain the final transform. The proposed registration framework was applied to 3D real-time fetal ultrasound images, thus covering the entire fetal anatomy for extended FoV imaging. Our method was applied to data from 5 patients, and compared to a conventional saliency point detection method (SIFT) in terms of computational time, quality of the point detection and registration accuracy. Our method achieved similar accuracy and similar saliency detection quality in \(<5\%\) the detection time, showing promising capabilities towards real-time whole-body fetal ultrasound imaging.

MR images of the fetus allow non-invasive analysis of the fetal brain. Quantitative analysis of fetal brain development requires automatic brain tissue segmentation that is typically preceded by segmentation of the intracranial volume (ICV). This is challenging because fetal MR images visualize the whole moving fetus and in addition partially visualize the maternal body. This paper presents an automatic method for segmentation of the ICV in fetal MR images. The method employs a multi-scale convolutional neural network in 2D slices to enable learning spatial information from larger context as well as detailed local information. The method is developed and evaluated with 30 fetal T2-weighted MRI scans (average age \(33.2\pm 1.2\) weeks postmenstrual age). The set contains 10 scans acquired in axial, 10 in coronal and 10 in sagittal imaging planes. A reference standard was defined in all images by manual annotation of the intracranial volume in 10 equidistantly distributed slices. The automatic analysis was performed by training and testing the network using scans acquired in the representative imaging plane as well as combining the training data from all imaging planes. On average, the automatic method achieved Dice coefficients of 0.90 for the axial images, 0.90 for the coronal images and 0.92 for the sagittal images. Combining the training sets resulted in average Dice coefficients of 0.91 for the axial images, 0.95 for the coronal images, and 0.92 for the sagittal images. The results demonstrate that the evaluated method achieved good performance in extracting ICV in fetal MR scans regardless of the imaging plane.

In this paper, voxel probability maps generated by a novel fovea fully convolutional network architecture (FovFCN) are used as additional feature images in the context of a segmentation approach based on deformable shape models. The method is applied to fetal 3D ultrasound image data aiming at a segmentation of the abdominal outline of the fetal torso. This is of interest, e.g., for measuring the fetal abdominal circumference, a standard biometric measure in prenatal screening. The method is trained on 126 3D ultrasound images and tested on 30 additional scans. The results show that the approach can successfully combine the advantages of FovFCNs and deformable shape models in the context of challenging image data, such as given by fetal ultrasound. With a mean error of 2.24 mm, the combination of model-based segmentation and neural networks outperforms the separate approaches.

3D ultrasound (US) is a promising technique to perform automatic extraction of standard planes for fetal anatomy assessment. This requires prior organ localization, which is difficult to obtain with direct learning approaches because of the high variability in fetus size and orientation in US volumes. In this paper, we propose a methodology to overcome this spatial variability issue by scaling and automatically aligning volumes in a common 3D reference coordinate system. This preprocessing allows the organ detection algorithm to learn features that only encodes the anatomical variability while discarding the fetus pose. All steps of the approach are evaluated on 126 manually annotated volumes, with an overall mean localization error of 11.9 mm, showing the feasibility of multi-organ detection in 3D fetal US with machine learning.

We propose a fully three-dimensional Convolutional Regression Network (CRN) for the task of predicting fetal brain maturation from 3D ultrasound (US) data. Anatomical development is modelled as the sonographic patterns visible in the brain at a given gestational age, which are aggregated by the model into a single value: the brain maturation (BM) score. These patterns are learned from 589 3D fetal volumes, and the model is applied to 3D US images of 146 fetal subjects acquired at multiple, ethnically diverse sites, spanning an age range of 18 to 36 gestational weeks. Achieving a mean error of 7.7 days between ground-truth and estimated maturational scores, our method outperforms the current state-of-art for automated BM estimation from 3D US images.

In this paper, we aim at proving the effectiveness of dictionary learning techniques on the task of retinal blood vessel segmentation. We present three different methods based on dictionary learning and sparse coding that reach state-of-the-art results. Our methods are tested on two, well-known, publicly available datasets: DRIVE and STARE. The methods are compared to many state-of-the-art approaches and turn out to be very promising.

Recognizing significant temporal changes in the thickness of the choroid and retina at an early stage is a crucial factor in the prevention and treatment of ocular diseases such as myopia or glaucoma. Such changes are expected to be among the first indicators of pathological manifestations and are commonly dealt using segmentation-based approaches. However, segmenting the choroid is challenging due to low contrast, loss of signal and presence of artifacts in optical coherence tomography (OCT) images. In this paper, we present a novel method for early detection of choroidal changes based on piecewise rigid image registration. In order to adhere to the eye’s natural shape, the regularization enforces the local homogeneity of the transformations in nasal-temporal (x-) and superior-inferior (y-) direction by penalizing their radial differences. We restrict our transformation model to anterior-posterior (z-) direction, as we focus on juvenile myopia, which correlates to thickness changes in the choroid rather than to structural alterations. First, the precision of the method was tested on an OCT scan-rescan data set of 62 healthy Asian children, ages 7 to 13, from a population with a high prevalence of myopia. Furthermore, the accuracy of the method in recognizing synthetically induced changes in the data set was evaluated. Finally, the results were compared to those of manually annotated scans.

We present a novel approach to automatically identify the corneal ulcer areas using fluorescein staining images. The proposed method is based on a deep convolutional neural network that labels each pixel in the corneal image as either ulcer area or non-ulcer area, which is essentially a two-class classification problem. Patch-based approach was employed; for every image pixel, a surrounding patch of size 19 × 19 was used to extract the RGB intensities to be used as features for training and testing. For the architecture of our deep network, there were four convolutional layers followed by three fully connected layers with dropout. The final classification was inferred from the probabilistic output from the network. The proposed approach has been validated on a total of 48 images using 5-fold cross-validation, with high segmentation accuracy established; the proposed method was found to be superior to both a baseline method (active contour) and another representative network method (VGG net). Our automated segmentation method had a mean Dice overlap of 0.86 when compared to the manually delineated gold standard as well as a strong and significant manual-vs-automatic correlation in terms of the ulcer area size (correlation coefficient = 0.9934, p-value = 6.3e-45). To the best of our knowledge, this is one of the first few works that have accurately tackled the corneal ulcer area segmentation challenge using deep neural network techniques.

Central serous retinopathy is a serious retinal disease. Retinal layer segmentation for this disease can help ophthalmologists to provide accurate diagnosis and proper treatment for patients. In order to detect surfaces in optical coherence tomography images with pathological changes, an automatic method is reported by combining random forests and a live wire algorithm. First, twenty four features are designed for the random forest classifiers to find initial surfaces. Then, a live wire algorithm is proposed to accurately detect surfaces between retinal layers even though OCT images with fluids are of low contrast and layer boundaries are blurred. The proposed method was evaluated on 24 spectral domain OCT images with central serous retinopathy. The experimental results showed that the proposed method outperformed the state-of-art methods.

Retinal image quality classification makes a great difference in automated diabetic retinopathy screening systems. With the increase of application of portable fundus cameras, we can get a large number of retinal images, but there are quite a number of images in poor quality because of uneven illumination, occlusion and patients movements. Using the dataset with poor quality training networks for DR screening system will lead to the decrease of accuracy. In this paper, we first explore four CNN architectures (AlexNet, GoogLeNet, VGG-16, and ResNet-50) from ImageNet image classification task to our Retinal fundus images quality classification, then we pick top two networks out and jointly fine-tune the two networks. The total loss of the network we proposed is equal to the sum of the losses of all channels. We demonstrate the super performance of our proposed algorithm on a large retinal fundus image dataset and achieve an optimal accuracy of 97.12%, outperforming the current methods in this area.

In order to realize the localization of optic disc (OD) effectively, a new end-to-end approach based on CNN was proposed in this paper. CNN is a revolutionary network structure which has shown its power in fields of computer vision like classification, object detection and segmentation. We intend to make use of CNN in the study of fundus images. Firstly, we use a basic CNN on which specialized layers are trained to find the pixels probably in OD region. Then we sort out candidate pixels furtherly via threshold. By calculating the center of gravity of these pixels, the location of OD is finally determined. The method has been tested on three databases including ORIGA, MESSIDOR and STARE. In totally 1240 images to be tested, the OD of 1193 are successfully located with the rate of 96.2%. Besides the accuracy, the time cost is another advantage. It takes only 0.93 s to test one image on average in STARE and 0.51 s in MESSIDOR.

Choroid neovascularization (CNV) usually causes varying degrees of irreversible retinal degradation, central scotoma, metamorphopsia or permanent visual lose. If early prediction can be achieved, timely clinical treatment can be applied to prevent further deterioration. In this paper, a CNV growth prediction framework based on physiological structure revealed in noninvasive optical coherence tomography (OCT) images is proposed. The method consists of three steps: pre-processing, CNV growth modeling and prediction. For growth modeling, a new combination model is proposed. The hyperelastic biomechanical model and reaction-diffusion model with treatment factor are combined through mass effect. For parameter optimization, the genetic algorithm is applied. The proposed method was tested on a data set with 6 subjects, each with 12 longitudinal 3-D images. The experimental results showed that the average TPVF, FPVF and Dice coefficient of 80.0 ± 7.62%, 23.4 ± 8.36% and 78.9 ± 7.54% could be achieved, respectively.

In this work, we propose a novel similarity measure for non-rigid retinal optical coherence tomography image registration called conditional correlation ratio (CCR). CCR calculates the correlation ratio (CR) between the moving and fixed image intensities, given a certain spatial distribution. The proposed CCR-based registration is robust to noise and less sensitive to the number of samples used to estimation the density function. Compared to mutual information (MI) and CR, both the quantitative indicators using Hausdorff distance (HD) and M-Hausdorff distance (MHD) and the qualitative indicator using checkerboard images show that CCR is more suitable to align the retinal OCT images.

Glaucoma is a highly threatening and widespread ocular disease which may lead to permanent loss in vision. One of the important parameters used for Glaucoma screening in the cup-to-disc ratio (CDR), which requires accurate segmentation of optic cup and disc. We explore fully convolutional networks (FCNs) for the task of joint segmentation of optic cup and disc. We propose a novel improved architecture building upon FCNs by using the concept of residual learning. Additionally, we also explore if adversarial training helps in improving the segmentation results. The method does not require any complicated preprocessing techniques for feature enhancement. We learn a mapping between the retinal images and the corresponding segmentation map using fully convolutional and adversarial networks. We perform extensive experiments of various models on a set of 159 images from RIM-ONE database and also do extensive comparison. The proposed method outperforms the state of the art methods on various evaluation metrics for both disc and cup segmentation.

The choroid plays a critical role in maintaining the portions of the eye responsible for vision. Specific alterations in the choroid have been associated with several disease states, including age-related macular degeneration (AMD), central serous chorioretinopathy, retinitis pigmentosa and diabetes. In addition, choroid thickness measures have been shown as a predictive biomarker for treatment response and visual function. Where several approaches currently exist for segmenting the choroid in optical coherence tomography (OCT) images of healthy retina, very few are capable of addressing images with retinal pathology. The difficulty is due to existing methods relying on first detecting the retinal boundaries before performing the choroidal segmentation. Performance suffers when these boundaries are disrupted or suffer large morphological changes due to disease, and cannot be found accurately. In this work, we show that a learning based approach using convolutional neural networks can allow for the detection and segmentation of the choroid without the prerequisite delineation of the retinal layers. This avoids the need to model and delineate unpredictable pathological changes in the retina due to disease. Experimental validation was performed using 62 manually delineated choroid segmentations of retinal enhanced depth OCT images from patients with AMD. Our results show segmentation accuracy that surpasses those reported by state of the art approaches on healthy retinal images, and overall high values in images with pathology, which are difficult to address by existing methods without pathology specific heuristics.

Glaucoma, a progressive and degenerative disease of the optic nerve, is the second leading cause of blindness worldwide. Mechanical deformation of the lamina cribrosa (LC) under high intraocular pressure (IOP) can lead to axonal death of optic nerve fibers. To explore the effect of pressure on the LC, we utilize an experimental setup where longitudinal 3D optical coherence tomography (OCT) images are acquired at different levels of IOP administered via a well-controlled external force. Structural changes are measured via image deformations which map all observed images simultaneously into a common coordinate space. These deformations encode local patterns of structural and volume change across the image sequence, resulting in quantification of the spatiotemporal deformation pattern of the LC due to variation of pressure. We also describe a 3D segmentation algorithm to restrict our deformation analysis separately to the beams or pores of the LC. A single case study demonstrates the potential of the proposed methodology for non-invasive in-vivo analysis of LC dynamics in individual subjects.

Blur is a significant problem in medical imaging which can hinder diagnosis and prevent further automated or manual processing. The problem of restoring an image from blur degradation remains a challenging task in image processing. Semi-blind deblurring is a useful technique which may be developed to restore the underlying sharp image given some assumed or known information about the cause of degradation. Existing models assume that the blur is of a particular type, such as Gaussian, and do not allow for the approximation of images corrupted by other blur types which are not easily incorporated into deblurring frameworks. We present an automated approach to image deconvolution which assumes that the cause of blur belongs to a set of common types. We develop a hierarchical approach with convolutional neural networks (CNNs) to distinguish between blur types, achieving an accuracy of 0.96 across a test set of 900 images, and to determine the blur strength, achieving accuracy of 0.77 across 1500 test images. Given this, we are able to reconstruct the underlying image to mean ISNR of 7.53.

Exudates in retinal images are one of the early signs of the vision-threatening diabetic retinopathy and diabetic macular edema. Early diagnosis is very helpful in preventing the progression of the disease. In this work, we propose a fully automatic exudate segmentation method based on the state-of-the-art residual learning framework. With our proposed end-to-end architecture the training is done on small patches, but at the test time, the full sized segmentation is obtained at once. The small number of exudates in the training set and the presence of other bright regions are the limiting factors, which are tackled by our proposed importance sampling approach. This technique selects the misleading normal patches with a higher priority, and at the same time avoids the network to overfit to those samples. Thus, no additional post-processing is needed. The method was evaluated on three public datasets for both detecting and segmenting the exudates and outperformed the state-of-the-art techniques.

In-vivo specular microscopy provides information on the corneal health state. The correlation between corneal nerve tortuosity and pathology has been shown several times. However, because there is no unique formal definition of tortuosity, reproducibility is poor. Recently, two distinct forms of corneal nerve tortuosity have been identified, describing either short-range or long-range directional changes. Using 30 images and their manual grading provided by 7 experts, we automatically traced corneal nerves with a custom computerized procedure and identified the combination of geometrical measurements that best represents each tortuosity definition (Spearman Rank Correlation equal to 0.94 and 0.88, respectively). Then, we evaluated both of these tortuosity indexes in 100 images from 10 healthy and 10 diabetic subjects (5 images per subject). A Linear Discriminant Analysis showed a very good capability (accuracy 85%) to differentiate healthy subjects from pathological ones by using both tortuosity indexes together.

In this paper, we propose an automatic technique for the assessment of retinal vessel caliber in fundus images using a fully automatic technique exploiting a multi-scale active contour technique. The proposed method is compared with the well-known semi-automated IVAN software and the Vampire width annotation tool. Experimental results show that our approach is able to provide fast and fully automatic caliber measurements with similar caliber measurement and comparable system error as the IVAN software. It will benefit the analysis of quantitative retinal vessel caliber measurements in large-scale screening programs.

We propose a method for detecting mechanically induced wrinkles that present themselves around the optic disc as a result of swelling. Folds and wrinkles have recently been found to be useful features for diagnosing optic disc swelling and for differentiating papilledema (optic nerve swelling due to raised intracranial pressure) from pseudopapilledema. A total of 22 patients were diagnosed with varying degrees and causes of optic disc swelling, with 3D spectral domain optical coherence tomography (SD-OCT) images obtained. The images were used to create fold-enhanced 2D images. Features were extracted pertaining to the orientation, Gabor responses, Fourier responses, and coherence to train a pixel-level classifier to distinguish between folds, vessels, image artifacts, and background. An area under the curve of 0.804 was achieved for the classification.