RadLex Terms and Local Texture Features for Multimodal Medical Case Retrieval

Clinicians searching through the large data sets of multimodal medical information generated in hospitals currently do not fully exploit previous medical cases to retrieve relevant information for a differential diagnosis. The VISCERAL Retrieval benchmark organized a medical case–based retrieval evaluation using a data set composed of patient scans and RadLex term anatomy–pathology lists from the radiologic reports. In this paper a retrieval method for medical cases that uses both textual and visual features is presented. It defines a weighting scheme that combines the RadLex terms anatomical and clinical correlations with the information from local texture features obtained from the region of interest in the query cases. The method implementation, with an innovative 3D Riesz wavelet texture analysis and an approach to generate a common spatial domain to compare medical images is described. The proposed method obtained overall competitive results in the VISCERAL Retrieval benchmark and could be seen as a tool to perform medical case based retrieval in large clinical data sets.