Research on GRU Neural Network Satellite Traffic Prediction Based on Transfer Learning

The satellite network traffic is affected by the periodic changes of the satellite network topology, the frequent switching of the satellite inter-satellite links, and the dynamic change of the inter-satellite link on–off relationship with time. The load of the satellite network traffic is adjacent to the geographical location of the satellite. Satellite network traffic has more complex and nonlinear characteristics [1]. To prevent network congestion and improve the utilization of network resources, reasonable network traffic management is especially important. The prediction of network traffic can grasp the changing characteristics and trends of network traffic in advance, to specify a reasonable and effective traffic management strategy to meet the requirements of users for quality of service (QoS) [2]. Therefore, it is of great practical significance to establish a high-precision traffic prediction model for satellite network.

The traditional autoregressive model (AR), moving average model (MA), autoregressive moving average model (ARMA), and autoregressive Integrated moving average model (ARIMA) can only express short correlation traffic characteristics [3]. With the continuous introduction of technologies such as neural networks and support vector machines, prediction models based on machine learning algorithms have emerged, such as artificial neural networks, least squares support vector machines (LSSVM), extreme learning machines (ELM), etc. [4]. The problem with the above algorithm is the lack of consideration of the temporal correlation of time series data, the limited prediction accuracy, and the satellite network traffic cannot be predicted effectively [5]. The recurrent neural network (RNN) is a deep neural network that introduces cyclic feedback [6]. Long short-term memory (LSTM) network is a special model of RNN. It can learn the long-term dependence between time series data and can effectively solve the gradient disappearance and gradient explosion problem in traditional RNN training process. However, the LSTM network introduces three types of gate structures and state space, resulting in greater time complexity [7]. To alleviate computing resources of satellite, reduce computational complexity, this paper proposes a GRU neural network, which simplifies the three gate structures in LSTM into two kinds of gate structures, the update gate and the reset gate, and combines the cell state and output into one state [8]. In this simplified way, it not only retains the LSTM’s ability to store long-term state, but also greatly reduce the computational complexity. GRU can greatly improve the training efficiency of the model and retain the effect like LSTM [9].

To further reduce the consumption of satellite computing resources, and to solve the problem of insufficient real-time data on the star and sufficient historical data, a transfer learning method is introduced. By learning accumulated knowledge from data from similar domains, the transfer learning approach facilitates the formation of predictive models from data in the target domain. [10]. At the same time, in order to reduce the complexity of online update parameters on the satellite, we abandon the traditional Stochastic Gradient Descent (SGD)-based training method and study the low- computational complexity of particle filtering (PF) online training. The method further determines the optimal parameters of the model, improves the accuracy of the model prediction, and reduces the training time of the model.

The GRU neural network retains the ability to remember long-term states by using update gates and reset gate structures, and greatly reduces computational complexity [11]. The GRU neural network diagram is below (Fig. 1).

Use the formula to express:

The square brackets indicate that two vectors are connected, and ⨂ is matrix elements multiplication. \({{\sigma }}\) is a sigmoid function whose output is between 0 and 1, indicating the update and forgetting degree of information. \(r_{t}\) indicates the update gate, which is used to determine how much information is saved to the next moment at the previous moment. The larger the value of the update gate, the more information from the previous moment is retained. \(z_{t}\) indicates the reset gate, which is used to determine the status information of the previous moment. Among them, the parameters we need to learn training are \(W_{r}\), \(W_{z}\), \(W_{{\tilde{h}}}\) and \(W_{o}\), the input of the output layer is \(y_{t}^{i} = W_{o} h\), and the output is \(y_{t}^{o} = \sigma \left( {y_{t}^{i} } \right)\).

To solve the problem of satellite network traffic prediction, this paper proposes a GRU neural prediction framework of network traffic based on transfer learning. The framework is mainly composed of three parts: data processing module, model training module, and model transfer module. The data processing module is mainly responsible for pre-processing the data, converting the continuous flow data into discrete flow data to meet the input requirements of the model. The model building module is the core of the traffic prediction framework. This paper proposes a model tuning method such as batch normalization and dropout. A low complexity training method of particle filter model is proposed. The model transfer module is another important model. It transfers a training model with large number of offline traffic data into online model of satellite to avoid the problem of insufficient online traffic data. Finally, the GRU neural network traffic prediction is constructed.

The key of particle filter algorithm is to determine the state transition equation and observation equation of the system [14]. For the traffic prediction model of GRU neural network, the discrete time value is the number of iterations of the model, and the state of each system is the optimal solution of the model. Equations (1)-(5) as the state transition equation of the system, the mean square error loss function (7) as the system Observe the equation. The training process of GRU neural network model based on particle filter algorithm is as follows:

First, a discrete system dynamic model is established. The mathematical model is expressed as follows:Where \(X_{t}\) is the system state variable, \(Z_{t}\) is the true observation of the system, \(v_{t}\) is the system noise, and \(e_{t}\) is the measurement noise of the system.

Particle initialization: Each particle is considered that having equal weight if the system state is unknown. The initial particle set is generated by sampling with probability density \({\text{p}}\left( {x_{0} } \right)\): \(\left\{ {x_{0}^{i} ,\frac{1}{N};i = 1,2, \ldots ,N} \right\}\).

Initialize system state: Calculate the network output value y according to the parameters of the GRU neural network and Eqs. (1)-(5). Set the minimum threshold of the system, \(N_{thr}\). Let the total number of particles in the particle filter algorithm be \(N\), the total number of iterations is \(tf\), set the end loss value \(l\), and randomly generate \(N\) particles according to the prior probability density \(p\left( {x_{0} } \right)\).

Importance sampling: When \(k = 1,2, \ldots ,N\), to avoid particle degradation, it is necessary to copy some of the particles with higher weight and remove the particles with lower weight.

(1) First, randomly extract N particles from a probability distribution function:

(2) Update the weight of the particles and normalize the particle weights:

According to the state transition equation \(p\left( {x_{k}^{i} |x_{k - 1}^{i} } \right)\), N particles are extracted from the initialized particle group. According to the observation Eq. (7) of the system, the matching value of all particles \(x_{k}^{i}\) is calculated, and the optimal particle and its corresponding optimal target y value are selected. The weight of the particle that does not satisfy the constraint is reset to zero. When the constraint is satisfied, according to the current observation \(y_{k}^{i}\) and the Eqs. (15) and (16) and normalized, updating the weight of the particle.

Resampling: Calculate the number of valid particles:

When \(\hat{N}_{eff} < N_{thr}\), re-sampling is performed according to \(p\left( {\tilde{x}_{k}^{j} = x_{k}^{i} } \right) = \tilde{x}_{k}^{j}\). We select the particles with larger weights for copying, and delete the smaller ones. A new set of generated particles is formed: \(\left\{ {x_{k}^{j} ,\frac{1}{N};{\text{j}} = 1,2, \ldots ,N} \right\}\).

State Estimation: Estimating System Status and Variance

Let k = k+1, continue the calculation, and judge whether the set loss value termination condition is satisfied.

The setting of \(\hat{N}_{eff}\) in the particle filter will directly determine the prediction accuracy of the model. According to the above steps, iterative iteration, so that the optimal state transition equation estimation can be obtained, and the final flow prediction value can be obtained.

This paper analyzes the characteristics of data of satellite network traffic. We proposes a prediction algorithm for GRU neural network traffic based on migration learning. In this paper, the construction process of the GRU neural network model and the model setting method are described in detail. The algorithm flow of the online training update method based on particle filter is given. What’s more, we adopt the transfer learning method to avoid the problem of insufficient online traffic data and reduce the consumption of satellite computing resources. The simulation results show that compared with FARIMA algorithm and SVM algorithm, the proposed algorithm has superior prediction accuracy. We verify that the particle update based online update method has low complexity and fast convergence speed. In short, the proposed traffic prediction algorithm has higher traffic prediction accuracy, lower computational complexity, faster convergence speed, and can effectively reduce satellite computing storage resources. It is a superior prediction algorithm for predicting satellite traffic.