Abstract
In this paper, we present a multi-texton representation method for medical image retrieval, which utilizes the locality constraint to encode each filter bank response within its local-coordinate system consisting of the k nearest neighbors in texton dictionary and subsequently employs spatial pyramid matching technique to implement feature vector representation. Comparison with the traditional nearest neighbor assignment followed by texton histogram statistics method, our strategies reduce the quantization errors in mapping process and add information about the spatial layout of texton distributions and, thus, increase the descriptive power of the image representation. We investigate the effects of different parameters on system performance in order to choose the appropriate ones for our datasets and carry out experiments on the IRMA-2009 medical collection and the mammographic patch dataset. The extensive experimental results demonstrate that the proposed method has superior performance.
Similar content being viewed by others
References
Madabhushi A, Agner S, Basavanhally A, Doyle S, Lee G: Computer-aided prognosis: Predicting patient and disease outcome via quantitative fusion of multi-scale, multi-modal data. Comput Med Imaging Graph 35(7–8):506–514, 2011
Zaidi H, Vees H, Wissmeyer M: Molecular PET/CT imaging-guided radiation therapy treatment planning. Acad Radiol 16(9):1108–1133, 2009
Marcus C, Ladam-Marcus V, Cucu C, Bouché O, Lucas L, Hoeffel C: Imaging techniques to evaluate the response to treatment in oncology: Current standards and perspectives. Crit Rev Oncol Hematol 72(3):217–238, 2009
Greenspan H, Pinhas AT: Medical image categorization and retrieval for PACS using the GMM-KL framework. IEEE Trans Inf Technol Biomed 11(2):190–202, 2007
Dhara AK, Mukhopadhyay S, Dutta A, Garg M, Khandelwal N: Content-based image retrieval system for pulmonary nodules: assisting radiologists in self-learning and diagnosis of lung cancer. J Digit Imaging 30(1):63–77, 2017
Camlica Z, Tizhoosh HR, Khalvatid F: Medical image classification via SVM using LBP features from saliency-based folded data. In: Proc IEEE Int Conf Mach Learn Appl, 2015. p. 128–132.
Tang Q, Liu Y, Liu H: Medical image classification via multiscale representation learning. Artif Intell Med 79:71–78, 2017
Lin Z, Brandt J: A local bag-of-features model for large-scale object retrieval. In: Proc Eur Conf Comput Vis, 2010, p. 294–308
Lehmann TM, Deselaers T, Schubert H, Guld MO, Thies C, Fischer B, Spitzer K: IRMA–a content based approach to image retrieval in medical applications. In: IRMA Int Conf, 5033: 911–912, 2006.
Puzicha J, Hofmann T, Buhmann JM: Non-parametric similarity measures for unsupervised texture segmentation and image retrieval. In: Proc IEEE Conf Comput Vis Pattern Recognit, 1997, pp 267–272
Arandjelovic R, Zisserman A: Three things everyone should know to improve object retrieval. In: Proc IEEE Conf Comput Vis Pattern Recognit, 2012, pp 2911–2918
Kumar A, Kim J, Cai W, Fulham M, Feng D: Content-based medical image retrieval: A survey of applications to multidimensional and multimodality data. J Digit Imaging 26(6):1025–1039, 2013
Gletsos M, Mougiakakou SG, Matsopoulos GK, Nikita KS, Nikita AS, Kelekis D: A computer-aided diagnostic system to characterize CT focal liver lesions: design and optimization of a neural network classifier. IEEE Trans Info Technol Biomed 7(3):153–162, 2003
Lee C, Chen S, Tsai H, Chung P, Chiang Y: Discrimination of liver diseases from CT images based on Gabor filters. In: Proc IEEE Int Symp Biomed Comput-based Med Syst, 2006
Nanni L, Brahnam S, Lumini A: A very high performing system to discriminate tissues in mammograms as benign and malignant. Expert Syst Appl 39(2):1968–1971, 2012
Manik V, Andrew Z: A statistical approach to material classification using image patch exemplars. IEEE Trans Pattern Anal Mach Intell 31(11):2032–2047, 2009
Jégou H, Douze M, Schmid C: Improving bag-of-features for large scale image search. Int J Comput Vis 87(3):316–336, 2010
Galaro J, Judkins AR, Ellison D, Baccon J, Madabhushi A: An integrated texton and bag of words Cclassifier for identifying anaplastic medulloblastomas. In: Conf Proc IEEE Eng Med Biol Soc, 2011, pp 3443–3446
Riaz F, Silva FB, Ribeiro MD, Coimbra MT: Invariant Gabor Texture Descriptors for Classification of Gastroenterology Images. IEEE Trans Biomed Eng 59(10):2893–2904, 2012
Li Y, Chen H, Rohde GK, Yao C, Cheng L: Texton analysis for mass classification in mammograms. Pattern Recognit Lett 52:87–93, 2015
Fernández-Carrobles MM, Bueno G, Déniz O, Salido J, García-Rojo M, Gonzández-López L: Frequential versus spatial colour textons for breast TMA classification. Comput Med Imaging Graph 42:25–37, 2015
van Gemert JC, Veenman CJ, Smeulders AWM, Geusebroek J: Visual word ambiguity. IEEE Trans Pattern Anal Mach Intell 32(7):1271–1283, 2010
Julesz B: Textons, the elements of texture perception, and their interactions. Nature 290:91–97, 1981
Leung T, Malik J: Representing and recognizing the visual appearance of materials using three-dimensional textons. Int J Comput Vis 43(1):29–44, 2001
Tang Q, Sang N, Liu H: Contrast-dependent surround suppression models for contour detection. Pattern Recogn 60:51–61, 2016
Xie J, Zhang L, You J, Shiu S: Effective texture classification by texton encoding induced statistical features. Pattern Recogn 48(2):447–457, 2015
Shotton J, Winn J, Rother C, Criminisi A: Textonboost for image understanding: Multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int J Comput Vis 81:2–23, 2009
Li X, Williams S, Bottema MJ: Constructing and applying higher order textons: Estimating breast cancer risk. Pattern Recogn 47(3):1375–1382, 2014
Petroudi S, Brady M: Breast density characterization using texton distributions. In: Conf Proc IEEE Eng Med Biol Soc, 2011, pp 5004–5007
Zhang L, Fisher M, Wang W: Retinal vessel segmentation using multi-scale textons derived from keypoints. Comput Med Imaging Graph 45:47–56, 2015
Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y: Locality-constrained linear coding for image classification. In: Proc IEEE Conf Comput Vis Pattern Recognit, 2010, pp 3360–3367
Zhang P, Wee C, Nieghammer M, Shen D, Yap P: Large deformation image classification using generalized locality-constrained linear coding. In: Proc Int Conf Med Image Comput Comput Assist Interv (MICCAI), 2013, pp 292–299
Yang J, Yu K, Gong Y, Huang T: Linear spatial pyramid matching using sparse coding for image classification. In: Proc IEEE Conf Comput Vis Pattern Recognit, 2009, pp 1794–1801
Lazebnik S, Schmid C, Ponce J: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proc. IEEE Conf. Comput. Vis. Pattern Recogn 2: 2169–2178, 2006.
He K, Zhang X, Ren S, Sun J: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916, 2015
Meij E, Trieschnigg D, Rijke MD, Kraaij W: Conceptual language models for domain-specific retrieval. Inform Process Manag 46(4):448–469, 2010
Tommasi T, Caputo B, Welter P, Guld MO, Deserno TM: Overview of the CLEF 2009 medical image annotation track. In: CLEF 2009 Workshop. Lect Notes Comput Sci 6242(85–93), 2010
Lehmann TM, Schubert H, Keysers D, Kohnen M, Wein BB: The IRMA code for unique classification of medical images. In: Proc SPIE-Med Imaging, 5033:440–451, 2003
Rahman MM, Desai BC, Bhattacharya P: Medical image retrieval with probabilistic multiclass support vector machine classifiers and adaptive similarity fusion. Comput Med Imaging Graph 32(2):95–108, 2008
Jiang W, Er G, Dai Q, Gu J: Similarity based online feature selection in content-based image retrieval. IEEE Trans Image Process 15(3):702–712, 2006
Avni U, Greenspan H, Konen E, Sharon M, Goldberger J: X-ray categorization and retrieval on the organ and pathology level, using patch-based visual words. IEEE Trans Med Imaging 30(3):733–746, 2011
Liu X, Tizhoosh HR, Kofman J: Generating binary tags for fast medical image retrieval based on convolutional nets and radon transform. In: Proc IEEE Int Joint Conf Neural Networks, 2016, pp 2872–2878
Unay D, Soldea O, Ozogür-Akyuz S, Cetin M, Ercil A: Medical image retrieval and automatic annotation: VPA-SABANCI at imageCLEF 2009. Working Notes for CLEF 2009 Workshop, 2009.
de Oliveira JEE, Lopes APB, Camara-Chavez G, de Araújo A, Deserno TM, Mammo SVD: A content-based image retrieval system using a reference database of mammographies. In: Proc IEEE Int Symp Biomed Comput-based Med Syst, 2009, pp 1–4
Crammer K, Singer Y: On the algorithmic implementation of multiclass kernel-based vector machines. J Mach Learn Res 2:265–292, 2001
Pinho E, Godinho T, Valente F, Costa C: A multimodal search engine for medical imaging studies. J Digit Imaging 30(1):39–48, 2017
Acknowledgments
The main image dataset used in this study is courtesy of the IRMA Group, Aachen, Germany, http://irma-project.org. This work was supported by the Fundamental Research Funds for the Central Universities (CZP17033).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Tang, Q., Yang, J. & Xia, X. Medical Image Retrieval Using Multi-Texton Assignment. J Digit Imaging 31, 107–116 (2018). https://doi.org/10.1007/s10278-017-0017-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10278-017-0017-z