Skip to main content
Erschienen in: International Journal of Computer Assisted Radiology and Surgery 1/2018

16.11.2017 | Original Article

A fully automatic end-to-end method for content-based image retrieval of CT scans with similar liver lesion annotations

verfasst von: A. B. Spanier, N. Caplan, J. Sosna, B. Acar, L. Joskowicz

Erschienen in: International Journal of Computer Assisted Radiology and Surgery | Ausgabe 1/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Purpose

The goal of medical content-based image retrieval (M-CBIR) is to assist radiologists in the decision-making process by retrieving medical cases similar to a given image. One of the key interests of radiologists is lesions and their annotations, since the patient treatment depends on the lesion diagnosis. Therefore, a key feature of M-CBIR systems is the retrieval of scans with the most similar lesion annotations. To be of value, M-CBIR systems should be fully automatic to handle large case databases.

Methods

We present a fully automatic end-to-end method for the retrieval of CT scans with similar liver lesion annotations. The input is a database of abdominal CT scans labeled with liver lesions, a query CT scan, and optionally one radiologist-specified lesion annotation of interest. The output is an ordered list of the database CT scans with the most similar liver lesion annotations. The method starts by automatically segmenting the liver in the scan. It then extracts a histogram-based features vector from the segmented region, learns the features’ relative importance, and ranks the database scans according to the relative importance measure. The main advantages of our method are that it fully automates the end-to-end querying process, that it uses simple and efficient techniques that are scalable to large datasets, and that it produces quality retrieval results using an unannotated CT scan.

Results

Our experimental results on 9 CT queries on a dataset of 41 volumetric CT scans from the 2014 Image CLEF Liver Annotation Task yield an average retrieval accuracy (Normalized Discounted Cumulative Gain index) of 0.77 and 0.84 without/with annotation, respectively.

Conclusions

Fully automatic end-to-end retrieval of similar cases based on image information alone, rather that on disease diagnosis, may help radiologists to better diagnose liver lesions.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Horton KM, Bluemke DA, Hruban RH, Soyer P, Fishman EK (1999) CT and MR imaging of benign hepatic and biliary tumors. Radio Gr 19:431–451 Horton KM, Bluemke DA, Hruban RH, Soyer P, Fishman EK (1999) CT and MR imaging of benign hepatic and biliary tumors. Radio Gr 19:431–451
2.
Zurück zum Zitat Pandey P, Lewis H, Pandey A, Schmidt C, Dillhoff M, Kamel IR, Pawlik TM (2017) Updates in hepatic oncology imaging. Surg Oncol 26:195–206CrossRefPubMed Pandey P, Lewis H, Pandey A, Schmidt C, Dillhoff M, Kamel IR, Pawlik TM (2017) Updates in hepatic oncology imaging. Surg Oncol 26:195–206CrossRefPubMed
3.
Zurück zum Zitat DeVita VT, Lawrence TSRS (2011) DeVita, Hellman, and Rosenberg’s cancer: principles and practice of oncology. Lippincott Williams & Wilkins, Philadelphia DeVita VT, Lawrence TSRS (2011) DeVita, Hellman, and Rosenberg’s cancer: principles and practice of oncology. Lippincott Williams & Wilkins, Philadelphia
4.
Zurück zum Zitat Assy N, Nasser G, Djibre A, Beniashvili Z, Elias S, Zidan J (2009) Characteristics of common solid liver lesions and recommendations for diagnostic workup. World J Gastroenterol 15:3217–27CrossRefPubMedPubMedCentral Assy N, Nasser G, Djibre A, Beniashvili Z, Elias S, Zidan J (2009) Characteristics of common solid liver lesions and recommendations for diagnostic workup. World J Gastroenterol 15:3217–27CrossRefPubMedPubMedCentral
5.
Zurück zum Zitat Oliver JH, Baron RL (1996) Helical biphasic contrast-enhanced CT of the liver: technique, indications, interpretation, and pitfalls. Radiology 201:1–14CrossRefPubMed Oliver JH, Baron RL (1996) Helical biphasic contrast-enhanced CT of the liver: technique, indications, interpretation, and pitfalls. Radiology 201:1–14CrossRefPubMed
6.
Zurück zum Zitat MacDonald SL, Cowan IA, Floyd R, Mackintosh S, Graham R, Jenkins E, Hamilton R (2013) Measuring and managing radiologist workload: application of lean and constraint theories and production planning principles to planning radiology services in a major tertiary hospital. J Med Imaging Radiat Oncol 57:544–550CrossRefPubMed MacDonald SL, Cowan IA, Floyd R, Mackintosh S, Graham R, Jenkins E, Hamilton R (2013) Measuring and managing radiologist workload: application of lean and constraint theories and production planning principles to planning radiology services in a major tertiary hospital. J Med Imaging Radiat Oncol 57:544–550CrossRefPubMed
7.
Zurück zum Zitat Zhao B, Tan Y, Bell DJ, Marley SE, Guo P, Mann H, Scott MLJ, Schwartz LH, Ghiorghiu DC (2013) Exploring intra- and inter-reader variability in uni-dimensional, bi-dimensional, and volumetric measurements of solid tumors on CT scans reconstructed at different slice intervals. Eur J Radiol 82:959–968CrossRefPubMed Zhao B, Tan Y, Bell DJ, Marley SE, Guo P, Mann H, Scott MLJ, Schwartz LH, Ghiorghiu DC (2013) Exploring intra- and inter-reader variability in uni-dimensional, bi-dimensional, and volumetric measurements of solid tumors on CT scans reconstructed at different slice intervals. Eur J Radiol 82:959–968CrossRefPubMed
8.
Zurück zum Zitat Suzuki C, Torkzad MR, Jacobsson H, Åström G, Sundin A, Hatschek T, Fujii H, Blomqvist L (2010) Interobserver and intraobserver variability in the response evaluation of cancer therapy according to RECIST and WHO-criteria. Acta Oncol (Madr) 49:509–514CrossRef Suzuki C, Torkzad MR, Jacobsson H, Åström G, Sundin A, Hatschek T, Fujii H, Blomqvist L (2010) Interobserver and intraobserver variability in the response evaluation of cancer therapy according to RECIST and WHO-criteria. Acta Oncol (Madr) 49:509–514CrossRef
9.
Zurück zum Zitat Dankerl P, Cavallaro A, Tsymbal A, Costa MJ, Suehling M, Janka R, Uder M, Hammon M (2013) A retrieval-based computer-aided diagnosis system for the characterization of liver lesions in CT scans. Acad Radiol 20:1526–34CrossRefPubMed Dankerl P, Cavallaro A, Tsymbal A, Costa MJ, Suehling M, Janka R, Uder M, Hammon M (2013) A retrieval-based computer-aided diagnosis system for the characterization of liver lesions in CT scans. Acad Radiol 20:1526–34CrossRefPubMed
10.
Zurück zum Zitat Müller H, Michoux N, Bandon D, Geissbuhler A (2004) A review of content-based image retrieval systems in medical applications-clinical benefits and future directions. Int J Med Inform 73:1–23CrossRefPubMed Müller H, Michoux N, Bandon D, Geissbuhler A (2004) A review of content-based image retrieval systems in medical applications-clinical benefits and future directions. Int J Med Inform 73:1–23CrossRefPubMed
11.
Zurück zum Zitat Akgül CB, Rubin DL, Napel S, Beaulieu CF, Greenspan H, Acar B (2011) Content-based image retrieval in radiology: current status and future directions. J Digit Imaging 24:208–22CrossRefPubMed Akgül CB, Rubin DL, Napel S, Beaulieu CF, Greenspan H, Acar B (2011) Content-based image retrieval in radiology: current status and future directions. J Digit Imaging 24:208–22CrossRefPubMed
12.
Zurück zum Zitat Liu Y, Zhang D, Lu G, Ma WY (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recognit Lett 40:262–282CrossRef Liu Y, Zhang D, Lu G, Ma WY (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recognit Lett 40:262–282CrossRef
13.
Zurück zum Zitat Grosky WI (2002) Narrowing the semantic gap—improved text-based web document retrieval using visual features. IEEE Trans Multimed 4:189–200CrossRef Grosky WI (2002) Narrowing the semantic gap—improved text-based web document retrieval using visual features. IEEE Trans Multimed 4:189–200CrossRef
14.
Zurück zum Zitat Deselaers T, Deserno TM, Müller H (2008) Automatic medical image annotation in ImageCLEF 2007: overview, results, and discussion. Pattern Recognit Lett 29:1988–1995CrossRef Deselaers T, Deserno TM, Müller H (2008) Automatic medical image annotation in ImageCLEF 2007: overview, results, and discussion. Pattern Recognit Lett 29:1988–1995CrossRef
15.
Zurück zum Zitat van Ginneken B, Schaefer-Prokop CM, Prokop M (2011) Computer-aided diagnosis: how to move from the laboratory to the clinic. Radiology 261:719–32CrossRefPubMed van Ginneken B, Schaefer-Prokop CM, Prokop M (2011) Computer-aided diagnosis: how to move from the laboratory to the clinic. Radiology 261:719–32CrossRefPubMed
16.
Zurück zum Zitat Ponciano-Silva M, Souza JP, Bugatti PH, Bedo MVN, Kaster DS, Braga RT V, Bellucci \({\hat{A}}\)D, Azevedo-Marques PM, Traina C, Traina AJM (2013) Does a CBIR system really impact decisions of physicians in a clinical environment? In: Proceedings of the 26th IEEE international symposium on computer based medical system CBMS 2013, pp 41–46 Ponciano-Silva M, Souza JP, Bugatti PH, Bedo MVN, Kaster DS, Braga RT V, Bellucci \({\hat{A}}\)D, Azevedo-Marques PM, Traina C, Traina AJM (2013) Does a CBIR system really impact decisions of physicians in a clinical environment? In: Proceedings of the 26th IEEE international symposium on computer based medical system CBMS 2013, pp 41–46
17.
Zurück zum Zitat An C, Rakhmonova G, Choi JY, Kim MJ (2016) Liver imaging reporting and data system (LI-RADS) version 2014: understanding and application of the diagnostic algorithm. Clin Mol Hepatol 22:296–307CrossRefPubMedPubMedCentral An C, Rakhmonova G, Choi JY, Kim MJ (2016) Liver imaging reporting and data system (LI-RADS) version 2014: understanding and application of the diagnostic algorithm. Clin Mol Hepatol 22:296–307CrossRefPubMedPubMedCentral
18.
Zurück zum Zitat Mitchell DG, Bruix J, Sherman M, Sirlin CB (2015) LI-RADS (Liver Imaging Reporting and Data System): Summary, discussion, and consensus of the LI-RADS Management Working Group and future directions. Hepatology 61:1056–1065CrossRefPubMed Mitchell DG, Bruix J, Sherman M, Sirlin CB (2015) LI-RADS (Liver Imaging Reporting and Data System): Summary, discussion, and consensus of the LI-RADS Management Working Group and future directions. Hepatology 61:1056–1065CrossRefPubMed
19.
Zurück zum Zitat Napel SA, Beaulieu CF, Rodriguez C, Cui J, Xu J, Gupta A, Korenblum D, Greenspan H, Ma Y, Rubin DL (2010) Automated retrieval of CT images of liver lesions on the basis of image similarity: method and preliminary results. Radiology 256:243–252CrossRefPubMedPubMedCentral Napel SA, Beaulieu CF, Rodriguez C, Cui J, Xu J, Gupta A, Korenblum D, Greenspan H, Ma Y, Rubin DL (2010) Automated retrieval of CT images of liver lesions on the basis of image similarity: method and preliminary results. Radiology 256:243–252CrossRefPubMedPubMedCentral
20.
Zurück zum Zitat Depeursinge A, Kurtz C, Beaulieu C, Napel S, Rubin D (2014) Predicting visual semantic descriptive terms from radiological image data: preliminary results with liver lesions in CT. IEEE Trans Med Imaging 33:1669–1676CrossRefPubMedPubMedCentral Depeursinge A, Kurtz C, Beaulieu C, Napel S, Rubin D (2014) Predicting visual semantic descriptive terms from radiological image data: preliminary results with liver lesions in CT. IEEE Trans Med Imaging 33:1669–1676CrossRefPubMedPubMedCentral
21.
Zurück zum Zitat Kumar A, Dyer S, Kim J, Li C, Leong PHW, Fulham M, Feng D (2016) Adapting content-based image retrieval techniques for the semantic annotation of medical images. Comput Med Imag Gr 49:37–45CrossRef Kumar A, Dyer S, Kim J, Li C, Leong PHW, Fulham M, Feng D (2016) Adapting content-based image retrieval techniques for the semantic annotation of medical images. Comput Med Imag Gr 49:37–45CrossRef
22.
Zurück zum Zitat Xu J, Napel S, Greenspan H, Beaulieu CF, Agrawal N, Rubin DL (2012) Quantifying the margin sharpness of lesions on radiological images for content-based image retrieval. Med Phys 39:5405CrossRefPubMedPubMedCentral Xu J, Napel S, Greenspan H, Beaulieu CF, Agrawal N, Rubin DL (2012) Quantifying the margin sharpness of lesions on radiological images for content-based image retrieval. Med Phys 39:5405CrossRefPubMedPubMedCentral
23.
Zurück zum Zitat Spanier AB, Joskowicz L (2014) Towards content-based image retrieval?: From computer generated features to semantic descriptions of liver CT scans. In: CLEF online work, notes. pp 438–447 Spanier AB, Joskowicz L (2014) Towards content-based image retrieval?: From computer generated features to semantic descriptions of liver CT scans. In: CLEF online work, notes. pp 438–447
24.
Zurück zum Zitat Spanier AB, Joskowicz L (2017) Automatic Atlas-free multi-organ segmentation of contrast-enhanced CT scans. In: Hanbury A, Müller H, Langs G (eds) Cloud-Based Benchmarking Med Image Anal. Springer, Berlin, pp 145–164CrossRef Spanier AB, Joskowicz L (2017) Automatic Atlas-free multi-organ segmentation of contrast-enhanced CT scans. In: Hanbury A, Müller H, Langs G (eds) Cloud-Based Benchmarking Med Image Anal. Springer, Berlin, pp 145–164CrossRef
25.
Zurück zum Zitat Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2011) SLIC superpixels compared to state-of-the-art superpixel methods. Pattern Anal Mach Intell IEEE Trans 34:2274–2282CrossRef Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2011) SLIC superpixels compared to state-of-the-art superpixel methods. Pattern Anal Mach Intell IEEE Trans 34:2274–2282CrossRef
26.
Zurück zum Zitat Cha SH, Srihari SN (2002) On measuring the distance between histograms. Pattern Recognit 35:1355–1370CrossRef Cha SH, Srihari SN (2002) On measuring the distance between histograms. Pattern Recognit 35:1355–1370CrossRef
27.
Zurück zum Zitat Marvasti NB, Kökciyan N, Türkay R, Yazici A, Yolum P, Üsküdarl S, Acar B (2014) ImageCLEF liver CT image annotation task 2014. In: CLEF (working notes), pp 329–340 Marvasti NB, Kökciyan N, Türkay R, Yazici A, Yolum P, Üsküdarl S, Acar B (2014) ImageCLEF liver CT image annotation task 2014. In: CLEF (working notes), pp 329–340
28.
Zurück zum Zitat Yolum P, Üsküdarl S, Acar B (2014) ImageCLEF liver CT image annotation task 2014. In: CLEF online work, notes, pp 329–340 Yolum P, Üsküdarl S, Acar B (2014) ImageCLEF liver CT image annotation task 2014. In: CLEF online work, notes, pp 329–340
29.
Zurück zum Zitat Pedregosa F, Grisel O, Weiss R, Passos A, Brucher M (2011) Scikit-learn?: Machine Learning in Python. J Mach Res 12:2825–2830 Pedregosa F, Grisel O, Weiss R, Passos A, Brucher M (2011) Scikit-learn?: Machine Learning in Python. J Mach Res 12:2825–2830
30.
Zurück zum Zitat Kalervo J, Kekäläinen J (2002) Cumulated gain-based evaluation of IR techniques. ACM Trans Inf Syst 20(4):422–446 Kalervo J, Kekäläinen J (2002) Cumulated gain-based evaluation of IR techniques. ACM Trans Inf Syst 20(4):422–446
31.
Zurück zum Zitat Spanier AB, Cohen D, Joskowicz L (2017) A new method for the automatic retrieval of medical cases based on the RadLex ontology. Int J Comput Assist Radiol Surg 12:471–484CrossRefPubMed Spanier AB, Cohen D, Joskowicz L (2017) A new method for the automatic retrieval of medical cases based on the RadLex ontology. Int J Comput Assist Radiol Surg 12:471–484CrossRefPubMed
32.
Zurück zum Zitat Chapelle O, Haffner P, Vapnik VN (1999) Support vector machines for histogram-based image classification. IEEE Trans Neural Netw 10:1055–1064CrossRefPubMed Chapelle O, Haffner P, Vapnik VN (1999) Support vector machines for histogram-based image classification. IEEE Trans Neural Netw 10:1055–1064CrossRefPubMed
33.
Zurück zum Zitat Echegaray S, Gevaert O, Shah R, Kamaya A, Louie J, Kothary N, Napel S (2015) Core samples for radiomics features that are insensitive to tumor segmentation: method and pilot study using CT images of hepatocellular carcinoma. J Med Imaging 2:41011CrossRef Echegaray S, Gevaert O, Shah R, Kamaya A, Louie J, Kothary N, Napel S (2015) Core samples for radiomics features that are insensitive to tumor segmentation: method and pilot study using CT images of hepatocellular carcinoma. J Med Imaging 2:41011CrossRef
34.
Zurück zum Zitat Järvelin K, Kekäläinen J (2002) Cumulated gain-based evaluation of IR techniques. ACM Trans Inf Syst 20:422–446CrossRef Järvelin K, Kekäläinen J (2002) Cumulated gain-based evaluation of IR techniques. ACM Trans Inf Syst 20:422–446CrossRef
Metadaten
Titel
A fully automatic end-to-end method for content-based image retrieval of CT scans with similar liver lesion annotations
verfasst von
A. B. Spanier
N. Caplan
J. Sosna
B. Acar
L. Joskowicz
Publikationsdatum
16.11.2017
Verlag
Springer International Publishing
Erschienen in
International Journal of Computer Assisted Radiology and Surgery / Ausgabe 1/2018
Print ISSN: 1861-6410
Elektronische ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-017-1687-1

Weitere Artikel der Ausgabe 1/2018

International Journal of Computer Assisted Radiology and Surgery 1/2018 Zur Ausgabe