Skip to main content
Top

29-06-2024

Visual Question Answer System for Skeletal Image Using Radiology Images in the Healthcare Domain Based on Visual and Textual Feature Extraction Techniques

Authors: Jinesh Melvin Y.I., Mukesh Shrimali, Sushopti Gawade

Published in: Annals of Data Science

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The Medical Imaging Query Response System is among the most challenging concepts in the medical field. It requires a significant amount of effort to organize and comprehend the various representations of the human body. Additionally, the system needs to be verified by users in the healthcare industry. With the aid of various images, including MRI scans, CT scans, ultrasounds, X-rays, PET-CT scans, and more, it may be possible to identify human health issues. It is anticipated to encourage patient participation and support clinical decision-making. As a result of the use of a number of characteristics that are inadequately matched to medical images and questions, technically, the VQA system in the healthcare domain is more complicated than in the common domain. The challenges were caused by the datasets, approaches, and models used for both visual and textual aspects. This can sometimes make it harder for clinical assistance to provide relevant answers. The proposed system will analyze current models and diagnose the problem in order to improve the medical visual question-answering system for recent datasets. The models that were compared to the model were convolutional neural networks (CNN), deep belief networks (DBN), recurrent neural networks (RNN), long short-term memory networks (LSTM), and bidirectional long short-term memory (BiLSTM). To assess the effectiveness of each model, the following measures should be used: Classification Accuracy, F-Classification, F-Measure, C-False Negative Rate (FNR), C-Positive Predictive Value, C-Precision, C-Recall, C-Sensitivity, and C-True Positive Rate (CTPR) With the objective of improving the performance of any dataset with accuracy and measures for both visual and textual features to get the right answers for given questions, the proposed system helps to recognize how ideal the existing models are and generates new models using the B12 FASTER Recurrent Neural Network (RNN) and Kai-Bi-LSTM. With questions and appropriate answers, the suggested model will assist in extracting the features of imported images and text.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Shi Y (2022) Advances in Big Data Analytics: theory, Algorithm and Practice. Springer, SingaporeCrossRef Shi Y (2022) Advances in Big Data Analytics: theory, Algorithm and Practice. Springer, SingaporeCrossRef
2.
go back to reference Tien JM (2017) Internet of things, real-time decision making, and artificial intelligence. Ann Data Sci 4(2):149–178CrossRef Tien JM (2017) Internet of things, real-time decision making, and artificial intelligence. Ann Data Sci 4(2):149–178CrossRef
11.
12.
go back to reference Pennington J, Socher R, Manning CD (2014) Glove: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP, A meeting of SIGDAT, a Special Interest Group of the ACL. ACL, Doha, Qatar, 1532–1543. https://doi.org/10.3115/v1/D14-1162 Pennington J, Socher R, Manning CD (2014) Glove: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP, A meeting of SIGDAT, a Special Interest Group of the ACL. ACL, Doha, Qatar, 1532–1543. https://​doi.​org/​10.​3115/​v1/​D14-1162
13.
go back to reference Herring W (2015) Learning radiology: recognizing the basics. Elsevier Health Sciences Herring W (2015) Learning radiology: recognizing the basics. Elsevier Health Sciences
15.
go back to reference Novelline RA, Squire LF (2004) Squire’s fundamentals of radiology. La Editorial, UPR Novelline RA, Squire LF (2004) Squire’s fundamentals of radiology. La Editorial, UPR
16.
go back to reference Jinesh Melvin YI, Gawade S, Palivela H (2021) Visual Question Answering using Data Mining Techniques for Skeletal Scintigraphy in medical domain - VQADMSS, 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS), Coimbatore, India. pp. 859–863, https://doi.org/10.1109/ICAIS50930.2021.9395936 Jinesh Melvin YI, Gawade S, Palivela H (2021) Visual Question Answering using Data Mining Techniques for Skeletal Scintigraphy in medical domain - VQADMSS, 2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS), Coimbatore, India. pp. 859–863, https://​doi.​org/​10.​1109/​ICAIS50930.​2021.​9395936
18.
37.
go back to reference Ben Abacha A, Datla VV, Hasan SA, Demner-Fushman D, Müller H (2020) Overview of the VQA-Med task at ImageCLEF 2020: Visual question answering and generation in the medical domain, in: CLEF 2020 Working Notes, CEUR-WS.org, Thessaloniki, Greece. https://medpix.nlm.nih.gov/home Ben Abacha A, Datla VV, Hasan SA, Demner-Fushman D, Müller H (2020) Overview of the VQA-Med task at ImageCLEF 2020: Visual question answering and generation in the medical domain, in: CLEF 2020 Working Notes, CEUR-WS.org, Thessaloniki, Greece. https://​medpix.​nlm.​nih.​gov/​home
38.
go back to reference Ben Abacha A, Sarrouti M, Demner-Fushman D, Hasan SA, Müller H (2021) Overview of the vqa-med task at imageclef 2021: Visual question answering and generation in the medical domain. In CLEF 2021 Working Notes, CEUR Workshop Proceedings. Conference and Labs of the Evaluation Forum, September 21–24, 2021, Bucharest, Romania. https://ceur-ws.org/Vol-2936/paper-87.pdf Ben Abacha A, Sarrouti M, Demner-Fushman D, Hasan SA, Müller H (2021) Overview of the vqa-med task at imageclef 2021: Visual question answering and generation in the medical domain. In CLEF 2021 Working Notes, CEUR Workshop Proceedings. Conference and Labs of the Evaluation Forum, September 21–24, 2021, Bucharest, Romania. https://​ceur-ws.​org/​Vol-2936/​paper-87.​pdf
44.
52.
go back to reference Cho K, Van Merri¨enboer B, G¨ulc¸ehre C¸, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using RNN encoder–decoder for statistical machine translation, in Proc. of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, pp. 1724–1734. https://doi.org/10.48550/arXiv.1406.1078 Cho K, Van Merri¨enboer B, G¨ulc¸ehre C¸, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014) Learning phrase representations using RNN encoder–decoder for statistical machine translation, in Proc. of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, pp. 1724–1734. https://​doi.​org/​10.​48550/​arXiv.​1406.​1078
54.
go back to reference Olson DL, Shi Y (2007) Introduction to business data mining. McGraw-Hill/Irwin, New York Olson DL, Shi Y (2007) Introduction to business data mining. McGraw-Hill/Irwin, New York
55.
go back to reference Shi Y, Tian YJ, Kou G, Peng Y, Li JP (2011) Optimization based data mining: theory and applications. Springer, BerlinCrossRef Shi Y, Tian YJ, Kou G, Peng Y, Li JP (2011) Optimization based data mining: theory and applications. Springer, BerlinCrossRef
Metadata
Title
Visual Question Answer System for Skeletal Image Using Radiology Images in the Healthcare Domain Based on Visual and Textual Feature Extraction Techniques
Authors
Jinesh Melvin Y.I.
Mukesh Shrimali
Sushopti Gawade
Publication date
29-06-2024
Publisher
Springer Berlin Heidelberg
Published in
Annals of Data Science
Print ISSN: 2198-5804
Electronic ISSN: 2198-5812
DOI
https://doi.org/10.1007/s40745-024-00553-0

Premium Partner