nach oben

Erschienen in:

2018 | OriginalPaper | Buchkapitel

Radiology Objects in COntext (ROCO): A Multimodal Image Dataset

verfasst von : Obioma Pelka, Sven Koitka, Johannes Rückert, Felix Nensa, Christoph M. Friedrich

Erschienen in: Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

This work introduces a new multimodal image dataset, with the aim of detecting the interplay between visual elements and semantic relations present in radiology images. The objective is accomplished by retrieving all image-caption pairs from the open-access biomedical literature database PubMedCentral, as these captions describe the visual content in their semantic context. All compound, multi-pane, and non-radiology images were eliminated using an automatic binary classifier fine-tuned with a deep convolutional neural network system. Radiology Objects in COntext (ROCO) dataset contains over 81k radiology images with several medical imaging modalities including Computer Tomography, Ultrasound, X-Ray, Fluoroscopy, Positron Emission Tomography, Mammography, Magnetic Resonance Imaging, Angiography. All images in ROCO have corresponding caption, keywords, Unified Medical Language Systems Concept Unique Identifiers and Semantic Type. An out-of-class set with 6k images ranging from synthetic radiology figures to digital arts is provided, to improve prediction and classification performance. Adopting ROCO, systems for caption and keywords generation can be modeled, which allows multimodal representation for datasets lacking text representation. Systems with the goal of image structuring and semantic information tagging can be created using ROCO, which is beneficial and of assistance for image and information retrieval purposes.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Weakly-Supervised Learning for Tool Localization in Laparoscopic Videos

Nächstes Kapitel Improving Out-of-Sample Prediction of Quality of MRIQC

https://github.com/razorx89/roco-dataset.

https://github.com/tensorflow/models/research/slim [last access: 09.04.2018].

Abadi, M., et al.: Tensorflow: a system for large-scale machine learning. In: Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation, Berkeley, CA, USA. USENIX Association (2016)

Bird, S., Klein, E., Loper, E.: Natural Language Processing with Python. O’Reilly (2009). http://www.oreilly.de/catalog/9780596516499/index.html

Bodenreider, O.: The unified medical language system (UMLS): integrating biomedical terminology. Nucl. Acids Res. 32(Database–Issue), 267–270 (2004). https://doi.org/10.1093/nar/gkh061CrossRef

García Seco de Herrera, A., Kalpathy-Cramer, J., Demner Fushman, D., Antani, S., Müller, H.: Overview of the ImageCLEF 2013 medical tasks. In: Working Notes of CLEF 2013 - Conference and Labs of the Evaluation forum. CEUR-WS Proceedings Notes, vol. 1179, Valencia, Spain, 23–26 September 2013 (2013)

García Seco de Herrera, A., Müller, H., Bromuri, S.: Overview of the ImageCLEF 2015 medical classification task. In: Working Notes of CLEF 2015 - Conference and Labs of the Evaluation Forum. CEUR-WS Proceedings Notes, vol. 1391, Toulouse, France, 8–11 September 2015 (2015)

García Seco de Herrera, A., Schaer, R., Bromuri, S., Müller, H.: Overview of the ImageCLEF 2016 medical task. In: Working Notes of CLEF 2016 - Conference and Labs of the Evaluation forum, Évora. CEUR-WS Proceedings Notes, vol. 1609, Portugal, 5–8 September 2016 (2016)

Koitka, S., Friedrich, C.M.: Optimized convolutional neural network ensembles for medical subfigure classification. In: Jones, G.J.F., et al. (eds.) CLEF 2017. LNCS, vol. 10456, pp. 57–68. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65813-1_5CrossRef

Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems, vol. 1, pp. 1097–1105. Curran Associates Inc., USA (2012)

LeCun, Y., Bengio, Y., Hinton, G.E.: Deep learning. Nature 521(7553), 436–444 (2015). https://doi.org/10.1038/nature14539CrossRef

10.

Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part V. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48CrossRef

11.

Müller, H., Kalpathy-Cramer, J., Demner-Fushman, D., Antani, S.: Creating a classification of image types in the medical literature for visual categorization. In: Proceedings of SPIE 8319, Medical Imaging 2012: Advanced PACS-Based Imaging Informatics and Therapeutic Applications, 83190P, 23 February 2012, vol. 8425, p. 194 (2012). https://doi.org/10.1117/12.911186

12.

Müller, H., et al.: Overview of the CLEF 2010 medical image retrieval track. In: CLEF 2010 LABs and Workshops, Notebook Papers. CEUR-WS Proceedings Notes, vol. 1176, Padua, Italy, 22–23 September 2010 (2010). http://ceur-ws.org/Vol-1176/CLEF2010wn-ImageCLEF-MullerEt2010.pdf

13.

Pelka, O., Friedrich, C.M.: FHDO biomedical computer science group at medical classification task of ImageCLEF 2015. In: Working Notes of CLEF 2015 - Conference and Labs of the Evaluation forum, Toulouse, France, 8–11 September 2015 (2015). http://ceur-ws.org/Vol-1391/14-CR.pdf

14.

Pelka, O., Friedrich, C.M.: Keyword generation for biomedical image retrieval with recurrent neural networks. In: Working Notes of CLEF 2017 - Conference and Labs of the Evaluation Forum. CEUR-WS Proceedings Notes, vol. 1866, Dublin, Ireland, 11–14 September 2017 (2017)

15.

Pelka, O., Nensa, F., Friedrich, C.M.: Adopting semantic information of grayscale radiographs for image classification and retrieval. In: Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2018). BIOIMAGING, vol. 2, Funchal, Madeira, Portugal, 19–21 January 2018, pp. 179–187 (2018). https://doi.org/10.5220/0006732301790187

16.

Porter, M.F.: Snowball: a language for stemming algorithms (2001). http://www.webcitation.org/6yci04ExR

17.

Rajpurkar, P., et al.: Chexnet: radiologist-level pneumonia detection on chest x-rays with deep learning. CoRR abs/1711.05225 (2017). https://arxiv.org/abs/1711.05225

18.

Roberts, R.J.: PubMed central: the GenBank of the published literature. Proc. Natl. Acad. Sci. U.S.A. 98(2), 381–382 (2001). https://doi.org/10.1073/pnas.98.2.381MathSciNetCrossRef

19.

Russakovsky, O.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-yMathSciNetCrossRef

20.

Soldaini, L., Goharian, N.: QuickUMLS: a fast, unsupervised approach for medical concept extraction. In: MedIR Workshop, SIGIR (2016)

21.

Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: Thirty-First AAAI Conference on Artificial Intelligence (2017). https://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14806

22.

Tommasi, T., Caputo, B., Welter, P., Güld, M.O., Deserno, T.M.: Overviewof the CLEF 2009 medical image annotation track. In: Multilingual Information Access Evaluation II. Multimedia Experiments - 10th Workshop of the Cross-Language Evaluation Forum, CLEF 2009, Corfu, Greece, 30 September–2 October 2009, pp. 85–93 (2009). https://doi.org/10.1007/978-3-642-15751-6_9

23.

Vinyals, O., Toshev, A., Bengio, S., Erhan, D.: Show and tell: lessons learned from the 2015 MSCOCO image captioning challenge. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 652–663 (2017). https://doi.org/10.1109/TPAMI.2016.2587640CrossRef

24.

Xu, Y., Mo, T., Feng, Q., Zhong, P., Lai, M., Chang, E.I.: Deep learning of feature representation with multiple instance learning for medical image analysis. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2014, Florence, Italy, 4–9 May 2014, pp. 1626–1630 (2014). https://doi.org/10.1109/ICASSP.2014.6853873

Titel: Radiology Objects in COntext (ROCO): A Multimodal Image Dataset
verfasst von: Obioma Pelka
Sven Koitka
Johannes Rückert
Felix Nensa
Christoph M. Friedrich
Verlag: Springer International Publishing
Buch: Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis
Print ISBN: 978-3-030-01363-9

Electronic ISBN: 978-3-030-01364-6

Copyright-Jahr: 2018
DOI: https://doi.org/10.1007/978-3-030-01364-6_20

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"