Mild Cognitive Impairment Prediction Using Facial and Speech Data

Mild cognitive impairment (MCI) represents a transitional stage between the cognitive decline associated with normal aging and more severe conditions such as dementia. Early diagnosis of MCI is crucial for effective healthcare intervention. However, current detection methods are often costly and time-consuming. This study introduces a multimodal fusion network (MFN) designed to predict MCI more efficiently. The proposed network utilizes dual-stream ResNets to process both facial and speech features. These features, extracted from the convolutional and subsampling layers of the ResNets, are subsequently fused in a fully connected layer to generate the final prediction. The dataset comprises a total of 52 participant videos, with an equal distribution: 26 videos from participants with normal cognitive function and 26 videos from participants diagnosed with MCI. Experimental results demonstrate the effectiveness of this approach, with an F1 score of 0.89 across test participants.