Application of sliding window technique for prediction of wind velocity time series

ANNs are of the most popular artificial intelligence techniques. They imitate the human brain systems and learn from examples. Applications of ANNs include but are not limited to clustering, regression analysis, estimating functions, fitness approximation, novelty detection, data processing as well as time series prediction. ANNs are capable of learning, memorizing and constructing relationships among data. An ANN consists of processing units, layers (input, output and hidden), neurons and transfer (activation) functions. ANN units are connected by weighted links that pass information and store the required knowledge in hidden layers. An input (x _j ) of the network travels thorough the connection and the link multiplies its strength by a weight (w _ij ) to generate x _j w _ij which is an argument of a transfer function (f). The transfer function produces an output as y _i  = f(x _j w _ij ), where i is an index of neurons in the hidden layer and j is an index of an input to the neural network [22]. During a training process, ANNs change weights to be trained for a minimum error and to achieve defined stopping criteria such as a considered error value, number of iterations, calculation time and validation limits. Multi-layer perceptron (MLP) networks are of the most common feed-forward estimator networks. These provide a robust architecture for learning nonlinear phenomenon. Feed-forward implies that neurons in continuous layers send their output (signals) forward and only forward connections exist [23]. MLP networks have been the most used architecture both in the renewable energy domain and in time series forecasting [24]. To predict values in MLP networks, a fixed number p of previous observed velocity values is considered as inputs of the network for each training process while the output is the forecasted values of the time series, which is the so-called the “sliding window technique” (see Fig. 2). The mathematical model for training the network can be expressed as:where P and Q are the number of inputs and hidden nodes, respectively, and j is the transfer function. Respectively, “w _j , j = 0, 1,…, Q” and “w _i,j , i = 1, 2,…, P & j = 1, 2,…, Q” are the weight vectors from the hidden layer to the output and the weights from the input to the hidden nodes. Also, w _0j is the weight for each output between the input and the hidden layer [25]. For network consistency and reducing calculation time, input data should be normalized. Hence, the utilized data of the current study are normalized before processing. Normalization can be performed as follows:

The performance of trained networks depends on their architecture. The number of hidden layers, the number of neurons, the type of utilized transfer functions, the size of training and testing samples and the learning algorithms are effective training parameters in addition to the utilized stopping criteria. To find the best structure of the network in this study, various network structures are evaluated. Finding the best value/type for each network parameter involves finding a value/type for that parameter which minimizes the prediction error. In the same line, network parameters and their evaluated values/types in the study are given in Table 2. As noted earlier, a fixed number p of previous observed velocity values are considered as inputs to the network in each training process while the output is the forecasted velocity values of the time series. Data distribution for training, validation and test procedures is considered as 70, 15 and 15 % of the entire data set, respectively. The computation processes include training (via the Levenberg–Marquardt (LM) algorithm [26]) and evaluation of the results (via the statistical error indices), which are performed in the MATLAB^® environment.

Also, the same data are provided to various radial basis function neural networks (RBFNNs) [27] for comparing their best outcome to the best performance yielded from the MLP networks trained by the LM algorithm. Regarding structure of the trained RBFNNs, a greater range of neuron numbers in the hidden layer is tested and compared to the MLP networks. To identify the best structure of the RBFNN, 0–25 neurons are examined in the hidden layer and the performances calculated for each structure are determined and compared in the results section.

The most common statistical indicators, including root mean square error (RMSE), absolute fraction of variance (R ²) and mean square error (MSE), are utilized with the forecasted data (v _f) and the actual measured data (v _m) to evaluate the performance of the proposed predictor systems. Table 3 shows the utilized statistical indicators accompanied by their relevant mathematical expressions.

The results of training various networks with different structures are presented in this section.

For MLP networks, Table 4 shows a summary of trained structures with p = 4 input data and the related obtained errors (RMSE and MSE) and R ² to find the best structure. Regarding the utilized statistical indices, RMSE provides information on the performance of the models, as it allows a term by term comparison of the actual deviation between the calculated and the measured values. The commonly used parameter, R ², is a measure of the strength of the relationship between values. It is a function of RMSE and the standard deviation. Hence, to evaluate the performance of the established networks, low error values (nearer to 0) and high R ² values (nearer to 1) are favorable for an efficient network.

According to the results, a network with 4-7-13-1 structure is chosen as the best MLP network of this study. This network utilizes 4 input data in the input layer with 7 and 13 neurons in its first and second hidden layers to predict the output. The activation functions of the first and the second relevant hidden layers of this network are Tansigmoid and Logsigmoid, respectively. The network utilizes a linear function in its output layer to transfer the data to the output.

Similarly, various RBFNN structures are examined to find the related outperforming architecture. The performances obtained for RBFNN structures with p = 4 input and 0–25 examined neurons in the hidden layer are summarized in Fig. 3.

To test the best structures of the trained RBF and MLP networks, the identified structure from each approach is separately deployed to the same unknown test data, which were not utilized in the training process. Table 5 shows the performance results related to the outperformed structures of MLP (4-7-13-1) and RBFNN (4-25-1), considering the same test data.

Figure 4 illustrates the agreement of the actual target values and the predicted outputs for the outperformed structure of RBFNN (4-25-1) after de-normalization of the data. Along the same line, Fig. 5 illustrates the outperformed structure of MLP (4-7-13-1).

Evaluation of the statistical indices demonstrates acceptable predicted outputs and also the flexibility of both approaches when large changes occur. However, as shown in Table 5, the 4-7-13-1-MLP network performs better than the 4-25-1-RBFNN network, according to its higher R ² value and lower MSE and RMSE values. Figure 6 illustrates a schematic structure of this selected network and its components.

For the outperformed structure of MLP network, applying the statistical indices with the predicted and the actual test data results in acceptable RMSE, MSE and R ² values (considering similar previous studies, including Refs. [9, 14, 15]) of 1.19, 1.43 and 0.85, respectively. As can be seen in the error histogram of this network (Fig. 7), the largest error values range between −1 and 1, which indicates acceptable overestimation and underestimation values. Also, the standard deviation of error is 1.1951 and the symmetric shape of the error histogram around the zero point (following a normal distribution pattern) ensures low error values (near zero) for the sum of overestimated (positive values) and underestimated (negative values) data, when the sum of predicted velocities in a given period is desired.