Non-local MRI upsampling
Introduction
In medical imaging, the image voxel size is limited by a number of factors such as imaging hardware, signal to noise ratio (SNR), time constraints or patient’s comfort. In many cases, the acquired voxel size has to be decreased to fit with a concrete resolution requirement. In such situations, interpolation techniques can be applied (Thévenaz et al., 2000, Lehmann et al., 1999).
Super resolution (SR) techniques have emerged as an alternative to effectively increase the resolution of the reconstructed data (Carmi et al., 2006). In MRI, SR techniques have been previously applied to increase image resolution on functional MRI (fMRI) (Kornprobst et al., 2003) and diffusion tensor imaging (DTI) studies (Peled and Yeshurun, 2001). Unfortunately, most of these techniques are based on the acquisition of multiple low resolution (LR) images with small shifts which is time consuming and therefore not adequate for typical clinical settings.
This may be the reason because common interpolation techniques, such as linear interpolation or spline-based methods, have been used extensively in the past to decrease voxel size and increase apparent data resolution. These techniques assume that the existing points in the LR image may be represented using some kind of generic functions (e.g., nth order polynomials in case of B-splines) with some smoothness assumptions. However, such techniques estimate new points assuming that the existing ones (in the LR images) have the same value in the HR images which is only valid within homogeneous regions. These methods do not take into account the fact that the actual LR voxel intensity does not corresponds to an ideal sampling but a non-ideal sampling where the voxel intensity represents the weighted average of the underlying HR voxels within the LR voxel. As a result, interpolated images are typically, blurred versions of its corresponding HR reference images.
As an alternative to multiple image SR techniques, single image SR techniques perform resolution enhancement using the expected degradation model (Zhang and Cham, 2008) or exploiting the normal pattern redundancy available on image data (Ebrahimi and Vrscay, 2007, Xin Li, 2008, Protter et al., 2009, Elad and Datsenko, 2009). These latter techniques have shown very competitive results compared to classical interpolation methods. As was done for denoising, where the NL-means demonstrated high abilities to denoise images (Buades et al., 2005), these patch-based interpolation techniques enable significant improvements over the classical methods.
The exemplar-based or patch-based methods take advantage of the self-similarity of the image. In fact, instead of acquiring several images of the same object to increase the image resolution, the patch-based approaches consider that is possible to extract similar information by using the non-local redundancy within a single image. As described in Ebrahimi and Vrscay (2007), while fractal-based approaches use self-similarity across scales, patch-based methods take benefit of self-similarity at the same scale. In both cases, the validity of self-similarity assumption has been verified for natural images in several domains such as compression (Fisher, 1995), denoising (Buades et al., 2005), or super resolution (Protter et al., 2009). Recently, the validity of the pattern-redundancy assumption has been also successfully verified for 2D and 3D MR images (Manjón et al., 2008, Coupé et al., 2008a, Coupé et al., 2008b).
As described by Ebrahimi and Vrscay (2007) and detailed by Elad and Datsenko (2009), the examples/patches can be used in several ways during image interpolation. The patches can be used to estimate the parameters that control the regularization procedure. For instance, the examples can be used to learn the prior parameters of energy functions involved in inverse problems such as Markov random field reconstruction (Roth and Black, 2005). The patches can also be used directly to reconstruct the image by averaging or best matching as in inpainting (Criminisi et al., 2004), texture synthesis (Efros and Leung, 1999) or denoising with the NL-means filter (Buades et al., 2005). More recently, hybrid methods using examples within an explicit regularization expression have been proposed (Elad and Datsenko, 2009, Xin Li, 2008). In the latter, the reconstruction obtained from examples is directly used in a regularization function within an optimization procedure with very good results.
The method proposed in this paper shares some characteristics with single image SR techniques based on self-similarity using a regularization expression (Xin Li, 2008, Protter et al., 2009). However, instead of achieving the optimization of an explicit expression using the noisy image, the proposed method is based on an iterative reconstruction–correction scheme involving a denoised version of the image. After a noise reduction stage, a patched-based non-local reconstruction is performed to take advantage of the self-similarity present in MR images. A coherence constraint is then applied in order to ensure that the reconstructed values fit with MR acquisition specificities (i.e. the fact that LR voxel intensity represents the information of a non-ideal sampling). These two steps are iteratively repeated within a coarse to fine scheme to achieve the image upsampling.
The structure of the paper is as follows. Section 2 describes the different steps of the proposed method. Section 3 proposes an extensive validation on synthetic and real MR datasets to demonstrate the ability of the proposed method to effectively increase voxel resolution in MRI by recovering high frequency information from the original LR data. To conclude, a discussion is presented in Section 4.
Section snippets
Materials and methods
In MR imaging, the common model assumes that LR voxels can be well modeled as the average of the corresponding HR voxel values plus some acquisition noise. This noise is Rician distributed in magnitude images but can be well-approximated within the imaged object as Gaussian distributed for typical clinical SNR values (SNR > 3) (Nowak, 1999, Gudbjartsson and Patz, 1995). Therefore, voxels in LR data y can be related to the corresponding x HR voxels using this expression for LR image formation:
Implementation details
The proposed method requires an initial interpolation to start the reconstruction process. In our experiments, we used the 3rd order B-spline interpolation as implemented in MATLAB 7.4 (Mathworks Inc.).
The described patch-based non-local reconstruction scheme has three free parameters. These include the radius v of the search volume Ω, the radius f of the 3D patch used to compute the similarity among voxels and the degree of filtering h. Parameters v and f were set to 3 and 1 as suggested in
Discussion
A new method for high quality MR image upsampling has been presented that enables recovery of some HR data information from LR data. The proposed method has been demonstrated, using synthetic and real data, to outperform classical interpolation methods.
The improved performance of the proposed methodology can be understood considering our two main contributions. First, the proposed method uses a physically plausible model (i.e. non-ideal sampling) which has been used to constrain the iterative
Acknowledgements
The authors want to thank Gracian Garcia-Martí for supplying the MR images used in this paper and for useful discussions about this work. This work has been partially supported by the Spanish Health Institute Carlos III through the RETICS Combiomed, RD07/0067/2001, the Spanish Ministry Science and Innovation through Grant TIN2008-04752 and by the Canadian grant Industry Cda (CECR)-Gevas-OE016.
References (28)
- et al.
Resolution enhancement in MRI
Magnetic Resonance Imaging
(2006) - et al.
MRI denoising using non local means
Medical Image Analysis
(2008) - Banerjee, J., Jawahar, C.V., 2008. Super-resolution of text images using edge-directed tangent field. In: The Eighth...
- et al.
A non local algorithm for image denoising
IEEE International Conference on Computer Vision and Pattern Recognition
(2005) - et al.
Self-similarity driven color demosaicking
IEEE Transactions on Image Processing
(2009) - et al.
Design and construction of a realistic digital brain phantom
IEEE Transactions on Medical Imaging
(1998) - et al.
An optimized blockwise non local means denoising filter for 3D magnetic resonance images
IEEE Transactions on Medical Imaging
(2008) - et al.
3D wavelet subbands mixing for image denoising
International Journal of Biomedical Imaging
(2008) - et al.
Region filling and object removal by exemplar-based image inpainting
IEEE Transactions on Image Processing
(2004) - et al.
Solving the inverse problem of image zooming using “self-examples”
Lecture Notes in Computer Science
(2007)
Texture synthesis by non-parametric sampling
IEEE International Conference on Computer Vision
Example-based regularization deployed to super-resolution reconstruction of a single image
The Computer Journal
Fractal Image Compression, Theory and Application
The Rician distribution of noisy MRI data
Magnetic Resonance in Medicine
Cited by (244)
New non-local mean methods for MRI denoising based on global self-similarity between values
2024, Computers in Biology and MedicineMulti-level feature extraction and reconstruction for 3D MRI image super-resolution
2024, Computers in Biology and MedicineA<sup>2</sup>OURSR: Adaptive adjustment based real MRI super-resolution via opinion-unaware measurements
2023, Computerized Medical Imaging and GraphicsRNLFNet: Residual non-local Fourier network for undersampled MRI reconstruction
2023, Biomedical Signal Processing and ControlSlice imputation: Multiple intermediate slices interpolation for anisotropic 3D medical image segmentation
2022, Computers in Biology and Medicine