Correlating Tumour Histology and ex vivo MRI Using Dense Modality-Independent Patch-Based Descriptors

Histological images provide reliable information on tissue characteristics which can be used to validate and improve our understanding for developing radiological imaging analysis methods. However, due to the large amount of deformation in histology stemming from resected tissues, estimating spatial correspondence with other imaging modalities is a challenging image registration problem. In this work we develop a three-stage framework for nonlinear registration between ex vivo MRI and histology of rectal cancer. For this multi-modality image registration task, two similarity metrics from patch-based feature transformations were used: the dense Scale Invariant Feature Transform (dense SIFT) and the Modality Independent Neighbourhood Descriptor (MIND). The potential of our method is demonstrated on a dataset of eight rectal histology images from two patients using annotated landmarks. The mean registration error was 1.80 mm after the rigid registration steps which improved to 1.08 mm after nonlinear motion correction using dense SIFT and to 1.52 mm using MIND.