Optimizing Feature Selection for Load Forecasting Using Multi-Head Attention Mechanism

Feature selection has a significant impact on the predictive performance of load forecasting models, with historical load features being particularly crucial. Existing studies typically use manual trial-and-error approaches to select historical load data... without dynamically assessing how significantly each time period affects predictions. This often results in the omission of certain key historical information during feature selection, thereby limiting forecasting performance. To address this issue, this study proposes a hybrid short-term load forecasting model for park-level applications based on CNN-GRU-MHA. In this model, a Convolutional Neural Network (CNN) is employed to extract local temporal features, a Gated Recurrent Unit (GRU) is utilized to capture long-term dependencies, and a Multi-Head Attention (MHA) mechanism dynamically assigns weights to historical load features from different time periods, highlighting key time-step information and enhancing feature representation and model interpretability. The proposed model is validated using cooling load data from a typical campus. Test results demonstrate that adding the Multi-Head Attention mechanism (MHA) improves prediction accuracy by 25.51% compared to traditional manual selection of input time periods. Further analysis of the MHA-assigned weights confirms that historical load features from different time periods all contribute to the forecasting results, with varying degrees of importance.