A data decomposition and attention mechanism-based hybrid approach for electricity load forecasting

Several physical, statistical, and data-driven approaches for estimating the energy demand have been proposed in the recent past. These methods have considered several factors such as thermal dynamics, environmental factors, behavioral information, and financial considerations for building accurate demand prediction models. On the whole, it has been found that AI has contributed significantly to this domain, as the traditional physics-driven methods have several shortcomings such as equations driven, less adaptability, less generalizability, and many more. Researchers in the past have introduced several data-driven models (including machine learning and deep learning approaches) to surpass the shortcomings of existing models.

Earlier studies in the domain investigated several well-known and popular regression models, including simple linear regression, multi-linear regression models, and auto-regressive models (AR, ARMA, ARIMA, SARIMA, SARIMAX) at the target prediction task. The performance benefits achieved have shown that the auto-regressive models work wells at understanding the hidden trends and seasonality patterns of the data. Also, some recent enhancements in the regressive models have shown promising results. In [9], the authors developed a logistic mixture vector auto-regressive model for capturing the intra-variations and inter-variations of the time-series patterns. The model has attained significant performance improvement by integrating clustering and forecasting through a probabilistic approach. Three variants of several existing methods (auto-regressive, smoothing, and neural networks) were proposed by Kychkin and Chasparis [7] in the year 2021. The experimental evaluation stated that seasonality-based regressive model performs superior to other models, thus achieving the best performance. Alotaibi et al. [29] proposed an approach to determine the demand response sizing of the renewable energy systems for remote locations. The authors employed meta-heuristic optimization strategy for achieving the reliable and accurate results. Madrid and Antonio [30] proposed a machine learning-based system to accurately estimate the short-term load. The authors suggested that the ensemble machines performs better at load forecasting than conventional models.

With the advancements in the neural models building domain (ANN, DNN, RNN, LSTM, and GRU), researchers worldwide have proved the prediction capabilities of these models in a variety of application domains [31‐33]. Some of the eminent research studies employing these models in the load estimation domain are discussed as follows: Bedi and Toshniwal [1] developed an LSTM model-based approach to forecast electricity demand. The authors captured seasonality patterns by integrating deep learning models with clustering analysis. The prediction benefits were well demonstrated on a dataset of UT Chandigarh, India. Mohammad and Kim [34] implemented neural models (ANN and RNN) to capture dynamic and uncertain variations of the energy load data. The authors explored several combinations of hyper-parameters to develop an optimal model for prediction. Huang et al. [35] introduced a probabilistic convolutional neural network-based method to estimate futuristic load. The approach involved implementing load range discretization to generate training samples (probabilistic). Mohammed et al. [36] proposed a novel regression-based approach to estimate electricity load of schools in Saudi Arabia. The authors investigated the effect of eleven external features on the electricity load using regression model. In this direction, Xu et al. [37] developed the Bayesian neural networks-based probabilistic approach to capture epistemic and aleatoric uncertainty of the electrical load patterns. Moradzadeh et al. [38] implemented a bi-directional LSTM network to capture historical and futuristic dependencies of the load series patterns. The developed model estimates demand at a micro-grid level, considering both residential households and commercial loads. Most of the research works in energy load forecasting are based on offline learning. In this context, Fekri et al. [39] introduced online learning-based adaptive approach to extract drifting patterns from the newly collected data. An online RNN is proposed to extract and learn new dependencies present in the data. In 2022, Khan et al. [40] combined echo network and CNN for delineating the renewable energy generation and consumption patterns. The approach aimed to establish an effective equilibrium between the consumers and the production units. Yazici et al. [41] worked on demonstrating the prediction capability of CNN models at short-term load estimation. A video pixel network based on 1D-CNN is implemented to estimate the futuristic load. The proposed network has shown improved ability compared to prediction performance estimation measures.

Hybrid techniques include amalgamating two or more statistical, machine learning or deep learning models for attaining better prediction accuracy. Many research studies have well investigated the performance benefits of the hybrid techniques. This section performs a deep analysis of dominant and recently introduced hybrid techniques in the energy load forecasting domain.

Mamun et al. [42] performed an in-detailed prediction performance analysis of various hybrid load demand prediction techniques, including ANN, SVM, genetic algorithms, and their variants. The authors explained the integrated analysis of several techniques by detailing the advantages, disadvantages, and difficulties associated with each approach. The energy consumption in a country may get affected by real-time pricing patterns. Considering this, a hybrid approach to forecast electricity demand has been proposed by Dai and Zhao (2020) [43]. The author combined the support vector model with feature selection and parameters optimization for the desired prediction task. Massaoudi et al. [44] proposed a novel ensemble approach combining boosting machines with extreme learning and multi-layer perceptron architecture for short-term electricity load forecasting. Five different parameters optimization strategies were employed to optimize the prediction results. The performance superiority of the ensemble approach is verified by evaluating prediction performance on two real-world datasets. Bashir et al. [45] developed a model integration prophet with LSTM. The non-linear residual patterns of the original dataset (predicted by the prophet model) were estimated by employing the LSTM model. The assessment results were obtained on an eight-year quarterly aggregated Elia grid dataset. Alsharekh et al. [46] provided a deep learning approach integrating convolutional network with sequential model for electricity load forecasting. The evaluation benefits of the approach are validated on a real-world energy dataset. Recent advancements in the electricity demand forecasting domain have shown that the combined benefits of various machine and deep learning models have helped in achieving better prediction accuracy. Convolution neural networks have been found very effective at extracting feature information from the different kinds of data. Also, the accuracy of sequential models (including RNN, LSTM, and GRU) has also been verified at the future timestamp prediction activities. In this context, Wu et al. [47] introduced an approach utilizing CNN and LSTM for short-term load forecasting. The performance evaluation of the approach demonstrated that it this approach has superior prediction accuracy than the conventional prediction methods. Bedi and Toshniwal [5] combined auto-encoder models with deep neural models for optimizing the prediction accuracy of long-term forecasting models.

The amalgamation of decomposition strategies, including fast Fourier transform, empirical mode decomposition, wavelet transform with neural models, has shown great potential in capturing the seasonal, non-linear and chaotic patterns of the load demand data. These decomposition techniques support capturing the data’s inherent complex time–frequency-based featural aspects. Zhang et al. (2018) [48] proposed a hybrid approach integrating auto-regressive models with EMD algorithm and fruit fly optimization. The authors emphasized the impact of social and natural factors on energy consumption. The usefulness of the approach is validated based on the simulation experiments. In year 2022, Huang et al. [49] integrated multivariate EMD algorithm with SVR and PSO technique for the day ahead peak demand forecasting. The empirical assessment is carried on a real-time peak power demand data of Victoria and North–South Wales, Australia. In a similar context, Bedi and Toshniwal [11] proposed an approach for load forecasting by integrating the EMD method with different deep neural models. The performance results are validated on the energy consumption dataset of UT Chandigarh, India. Zang et al. [50] proposed a two-stage preprocessing (including decomposition and reconstruction mechanism), LSTM and attention procedure to forecast day-ahead electricity demand of residential individuals. Lee and Cho [13] performed a comparative assessment of several machine learning, deep learning, and hybrid methods at the time-series prediction task. From the analysis results on Korea’s peal load performance dataset, the authors concluded that the hybrid models provide significant improvement than the traditional machine and deep learning models. Sekhar et al. [23] proposed a hybrid approach combining convolutional approach with Bi-LSTM architecture for short-term load forecasting. The proposed approach parameters selection is performed using the gray wolf optimization approach. Ghimire et al. [51] provided a hybrid approach utilizing LSTM architecture in amalgamation with decomposition technique. In a similar context, a load forecasting technique based on LSTM was introduced by Grandon et al. [52]. The evaluation of the approach was performed on a real-time electricity demand dataset of Ukraine. These recently introduced approaches evaluation results’ on the real-time datasets validated that the hybrid approaches perform much better than the conventional prediction approaches. However, these existing techniques lack at several crucial aspects such as noise handling, abrupt variations capturing, and complexity. To resolve these shortcomings, the present work introduces a workflow implementing data decomposition, attention, and Bi-GRU architecture for electricity load forecasting.

Time-series data are prone to noise (errors and non-standard examples), erroneous readings, missing data, inconsistencies, and redundant information that are often distributed across multiple heterogeneous sources. These data inconsistencies are the significant factors that reduce the data quality and consequently affect the quality and reliability of results. In this context, preprocessing techniques are commonly employed to enhance data quality and extract meaningful information that can be more effectively interpreted. The two major phases involved in the data preprocessing stage are discussed as follows:

Time-series data collected by means of sensors and devices are highly vulnerable to noise, outliers, or abnormal variations. The existence of such disruption or inconsistencies in the data may impact the overall analysis and prediction process. There are several means to mitigate the effect of noise in the data, i.e., by collecting larger sample space, series smoothing, sensors, or apparatus improvement or many more. In the current research, we have adopted smoothing to reduce the impact of noise on load time-series variations patterns. There exist several smoothing strategies that can be employed to improve the data, namely simple or running average smoothing, weighted average smoothing, Gaussian smoothing, and exponential smoothing [54]. The average smoothing methods consider each timestamp observation as a weighted sum of previous observations estimated by a user-defined window. These average smoothing methods have several shortcomings such as finite width window, handling outside range values, etc. Hence, we have implemented Gaussian smoothing for removing noise present in the load series dataset by convolving each input time series with a Gaussian function at timestamp \(t_i\) is defined as (with width b representing the smoothing parameter) [23, 55]The convolution function involves utilizing the Gaussian kernel function/curve to weigh the load timestamp observations present in the input dataset. Then, we proceed by calculating the new observation using the weighted average of k data points returned by kernel function. Mathematically, it can be presented aswhere \(X_{t(i)}\) denotes the input series, \(Y_t'\) represents the smoothed value. For an example, the input time series and the corresponding smoothened time series are demonstrated in Fig. 2. From the figure, it can be observed that the new smoothened load time series has less noise. The Gaussian smoothing filter has attuned the large dips and spikes very effectively. An important hyper-parameter while applying smoothing is to choose the value of alpha/b. Large value may lead to data loss by losing important spikes and dips of the non-linear time-series dataset. The value of this parameter has been estimated by random research technique.

In signal processing, decomposing a signal into multiple components is a potential technique, effectively utilized across several application domains. The specific application domains include utilizing decomposition strategies for reducing the amount of data to be transferred or stored (compression), eliminating unpredictable components, namely noise and outliers and extracting relevant information through filtering and retrieval. Several data decomposition techniques have been proposed in the past for both discrete and continuous time domains. Examples include the sine and cosine transforms, Fourier transform, wavelet decomposition, and mode decomposition (EMD) [56, 57]. From the exhaustive literature analysis, it has been identified that EMD provides more accurate results than other decomposition techniques [58], such as Fourier, wavelet decomposition, and many more.

Empirical mode decomposition (EMD) is a method of decomposing time–frequency series, whose initial proposal is to deal with signals governed by non-linear and non-stationary processes. The method was developed and introduced by Huang et al. [59] and it works by decomposing the input series into several orthogonally based functions called intrinsic mode functions (IMF). Each IMF obtained by decomposition highlights the local characteristics of the data. The lowest-frequency IMF component is the residual term, which represents the original signal’s general trend or mean value. The sum of all the IMFs with the residual recompose the original series, according to the following equation:Although the EMD technique has been widely employed, the EMD lacks sufficient theoretical basis, and its susceptibility to modal aliasing causes many uncertainties in the decomposition results. Despite these limitations, the EMD technique showcases versatility in decomposing a time domain signal into a set of nearly and complete orthogonal basis functions (i.e., the IMFs), each having the same length of the original signal, but exhibiting varying frequencies. Hence, enhanced versions of EMD have been developed to address the existing shortcomings. One such method is EEMD (ensemble empirical mode decomposition) introduced by Wu et al. [60]. The core idea behind this noise-assisted data analysis approach is to fuse the actual time-series information with noise so that even if different sources collect the same process data that has different characteristics, its overall mean should be close to the real-time series. To extract the real signal from the data, a method incorporating multiple sets of white noise sequences with limited amplitude to the original sequences is introduced. Subsequently, each signal is then decomposed separately, and the mean value of the corresponding components is considered as the real component. Additionally, the introduction of noise into the signal selected for decomposition has been proposed as a strategy to the mitigate the problem of mode mixing. This solution has led to the development of new variants of the EMD, such as the ensemble empirical mode decomposition (EEMD) [60], and the complete EEMD with adaptive noise known as complementary ensemble empirical mode decomposition with assisted noise (CEEMDAN) [61]. The current study adopts CEEMDAN data decomposition technique to overcome the shortcomings of existing data decomposition techniques. The CEEMDAN method is employed due to its capability to rectify the modes mixing problem. Moreover, the method also effectively suppresses the residual noise in the IMFs, thus leading to a superior signal decomposition and reconstruction (with minimal reconstruction error). The following steps briefly summarize the procedure of decomposition by complementary ensemble empirical mode with assisted noise:

This section describes the architectural explanation of the proposed load demand forecasting strategy. The overall section comprises two sub-phases. The first phase provides a comprehensive description, working, and suitability of the proposed attention-based bi-directional gated sequential neural model to the problem under study. In the second phase, the working of the proposed prediction model amalgamating the data decomposition, attention mechanism, and deep neural network models is summarized.

The current study collects and analyzes the real-time electricity consumption dataset of five southern states: Andhra Pradesh, Karnataka, Kerala, Tamil Nadu, and Puducherry. The southern states are known to be the key energy consumers, so targeting these states will help in finding/defining better energy management strategies for the country India. For an in-depth analysis of energy usage trends and patterns, the dataset for these states is collected over a period of seven years, starting from January 2014. The timestamp observations have been recorded on a per-day basis. Moreover, the geographical details of the five states, along with the statistical description of energy consumption usage patterns, are provided in Table 1.

This step entails applying the preprocessing “Data Preprocessing (Step-II)” and smoothing methods “Gaussian Smoothing (Step-III)” on the dataset collected for the five southern Indian states. There were very few missing values present in the dataset for each state. To address this, the mean values considering the next five days, corresponding to the same period of all seven years, are utilized to fill in these missing values. Since the present study considers only the historical load observations for estimating the future load, there is no need for the applicability of data reduction techniques. However, in context of data transformation, the normalization technique defined in eq. 1 is applied on the historical observations.

After data preprocessing, the time series representing each state energy usage pattern is fed to the smoothing module. The aim here is to remove the existence of any noisy and abnormal variations present in the usage profiles. The current study applied a Gaussian kernel to perform weighted smoothing on the data. For better understanding, an example figure demonstrating the application of Gaussian smoothing on time-series data of the ‘Puducherry’ state is shown in Fig. 2 From the figure, it is identified that the Gaussian smoothing aims to minimize the impact of any irrelevant or sudden observation patterns that may be present in the input energy consumption patterns data. This, in turn, helps to smoothen/amplify the critical variations patterns which the learning models can easily capture.

The next step after applying the smoothing technique is to decompose the time series corresponding to each state into several IMF components. For this, the CEEMDAN techniques discussed in “Data decomposition (Step-IV)” is employed. An example demonstration of the decomposition results achieved using CEEMDAN on the ‘Andhra Pradesh’ time series is provided in Fig. 3. The figure shows several IMFs components obtained for the input time series. Furthermore, from the figure, it is evident that the decomposition method efficiently segregates the time series into components with varying magnitude and inter IMF non-linear patterns, thus reducing the non-linearity and abrupt variations profiles in the raw input data.

The IMF components resulting from the above steps are fed as an input to the attention based bi-directional gated neural models. As discussed in the subsection, this step includes building a separate model for each IMF component generated for a southern state. The process is repeated for all the states present in the input dataset. Furthermore, to validate the effectiveness and reliability of our proposed work, advanced deep neural models (LSTM, GRU, and Bi-GRU) are also developed for state-level load estimation. The comparative evaluation outcomes of these models are listed in the Tables 3, 4, and 5. The tables represent the comparative results of the proposed approach against the benchmark approaches on the dataset of the southern states, namely Andhra Pradesh, Karnataka, Kerala, Tamil Nadu, and Puducherry, respectively. The comparative evaluation uses three widely known performance parameters: root mean squared error (RMSE), mean absolute percentage error (MAPE), and mean absolute error (MAE). A detailed description of these parameters is given as follows:

A critical factor to consider while developing any neural-based solution to a problem is to define the most-suitable value of model hyper-parameters. In the current study context, the identified optimal values corresponding to the proposed model hyper-parameters are outlined in Table 2. The results observed for the best combination of hyper-parameters are presented in Tables 3, 4, and 5. From analyzing the evaluation results, several critical observations are deduced:

This section includes depicting the predicting outcomes of the proposed approach on the input load demand dataset belonging to different states. The aim here is to assess and demonstrate the reliability and prediction capability of the proposed approach at capturing the non-linear and chaotic variations of the energy consumption dataset. To achieve this, the predicted timestamp values for different states generated by the proposed approach are plotted against their respective actual observation values in Fig. 4. In Fig. 4, the x-axis represents the samples and the y-axis denotes the energy consumption value. The actual observations are represented using the black color, while the predicted values on the training and testing dataset are colored in ‘blue’ and ‘purple’ color, respectively. From the Fig. 4, it is observed that the proposed approach accurately quantifies/captures the energy demand variations present in the dataset of different southern states. Moreover, the approach work very well at estimating the peak and abrupt energy demand requirement. Hence, from the comprehensive analysis, it can be stated that the approach can be reliably employed to accurately forecast energy demand in the real-world scenarios.

From the experimental evaluation of the proposed approach on five datasets corresponding to different southern states of India, it has been well validated that the proposed approach outperforms existing state-of-the-art approaches. These results are verified based on several different evaluation measures. A few critical observations are drawn from the experimental evaluation of the proposed approach. First, from the experimental evaluation, it has been observed that the models trained through the proposed approach attained saturation earlier than the conventional deep learning-based model-building strategy. The reason behind this is the inclusion of attention and smoothing mechanisms, which helped the proposed approach better learn from the critical weighted data dependencies identified through the attention module. The energy demand time-series data are highly complex, non-linear, and carries intrinsic data dependencies. In this direction, the decomposition procedure divides the complex non-linear energy demand patterns into several distinct, less-complex patterns, which has helped the proposed module easily capture the non-linear chaotic variations present in the data. This has resulted in improved accuracy of the proposed approach. Lastly, the prediction model’s performance is highly impacted by noise and unknown factors introduced by human errors or malfunctioning. In this context, introducing Gaussian smoothing to the proposed approach has reduced the impact of noise present in the collected dataset.