Hybrid attention network-based students behavior data analytics framework with enhanced capuchin search algorithm using multimodal data

Adaptive learning activities have a major impact on the achievement of a student, and it is helpful to identify how much effort they have to put forth. In today’s adaptive learning environments, elucidating and capturing student behavior and correlating it with academic success holds the potential for promptly giving students actionable and proactive feedback. In every learning situation, drawing conclusions about a variety of behavioral aspects made easier learning performance predictions by combining advanced deep learning algorithms with extensive learner data. However, research on identifying cooperation impasses is needed to increase the academic efforts on creating multimodal modeling approaches to analyze collaborative issues. This paper presents a new student behavior analysis model from multi-modal data like audio, video, text, face, and eye tracking. Secondly, the deep features from the multi-modal student’s behavior data are retrieved by using the deep learning structure, where 1D-Convolutional Neural Network (CNN), 3D-CNN, Transformernet and Autoencoder are adopted to extract the deep features from the audio, video, text and other physiological data, respectively. After extracting deep features, the behavior of the students is analyzed with the usage of the newly developed Hybrid Attention Network (HANet). In this HAN, deep learning structures like Temporal Convolutional Network (TCN) and Bi-directional Long Short Term Memory (Bi-LSTM) are incorporated, and the parameters present in this hybrid network are tuned by using the Enhanced Capuchin Search Algorithm (ECSA). The experimental results are validated with the conventional multi-modal data analytics model for student behavior with the help of positive and negative performance measures.