Hybrid CNN-LSTM with Attention Mechanism for Medical Visual Question Answering

This chapter explores the development and evaluation of a hybrid CNN-LSTM model with an attention mechanism for medical visual question answering (MedVQA). The model integrates convolutional neural networks (CNNs) for image feature extraction and long short-term memory (LSTM) networks for question encoding, enhanced by an attention mechanism to focus on relevant image regions. The study evaluates the model on two datasets: ImageCLEF VQA-MED 2019 and ImageCLEF VQA-RAD 2019, which include a variety of medical images such as X-rays, MRI, and CT scans, along with corresponding questions and answers. The results demonstrate the model's ability to achieve high training accuracy and reasonable validation accuracy, indicating its potential to improve clinical decision-making and healthcare outcomes. The chapter also discusses the challenges of class imbalances and overfitting, as well as recommendations for future improvements, such as using regularization techniques and collecting more balanced data. The findings highlight the model's effectiveness in handling complex medical question structures and its potential to transform healthcare by providing accurate and timely answers to medical queries.