Recurrent neural network model for high-speed train vibration prediction from time series

Deep learning systems are widely applied in many fields of modern technology due to high precision and many possible applications. Intelligent transportation is related to the variety of developments in technology. We can read about many interesting models used to predict or simulate vibrations. In [8] was proposed a model of vibration analysis for elements joined by friction. Neural networks were used to predict ground vibration in [3]. As a result, warning system for geological activities was developed. Similarly in [4] was presented how to use heuristic model to help on prediction of blast-produced ground vibration. A model composed for open-pit mine vibrations prediction presented in [12] was also based on neural network.

Deep learning models are very often used in diagnostic purposes for variety of technical systems. In [11] was presented how to evaluate displacement from data to improve transportation systems. Model presented in [5] was developed to help on fault detection in residual generator from collected data. There are also very interesting models of machine learning devoted to the topic of vibration analysis in means of transport. In [10] was presented an idea to evaluate vibration of high-speed railway by using deep learning. The model proposed in [13] was oriented on effect of vibrations that come from trains to urban areas. This topic was explored in [17] to show various aspects of such vibrations. Results presented in [21] described how energy consumption analysis may help in prediction of interior noise in a high speed. A wide spectrum of numerical experiments for various urban trains was presented in [7]. In [18] was presented an analytical approach to use neural networks for simulation of dynamical elements in moving vehicles. Among models used in analysis of time series, Recurrent Neural Networks (RNN) are presented in many efficient applications. An idea discussed in [20] has been developed for application of RNN in traffic noise prediction. Results presented in [6] show how to model RNN to infer global temporal structure. Presented results show that this type of neural networks can well adapt to variety of examples. In [19] was discussed efficiency of RNN in stochastic time series analysis, while results of [16] show solution to improve long time series processing with fuzzy granulation.

In this article, we want to discuss application of deep learning model to predict potential vibrations of high-speed trains from time series of recorded travels. In our work, we have prepared a novel lightweight solution for predicting the future train vibrations using time series data provided as an input. What is more, the proposed model performs not only well in terms of training and evaluation times but also in high Accuracy terms. Our proposed prediction model is developed using Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM) neurons. We have examined which training model would help to better train this system, and as a result, we propose to use NAdam training algorithm. Tests were also done for different configurations of the developed classifier. We have tested how many readings in a time proposed classifier can need to work with highest potential efficiency. We have also tested how long would it be necessary to train the classifier to work with assumed efficiency. Because we are working with time series data, we have decided to use LSTM neurons in our architecture as they have the best ability to remember past values and thus predicting trends. Other neuron types are ineffective because of great regression in performance in both training time and Accuracy standpoint. Results of our findings are compared and discussed to draw conclusions for future work on efficient predictors of long time series of sensor readings.

The research in our project is oriented on prediction of possible vibrations in high-speed trains by using deep neural network model. We have developed a model of Recurrent Neural Network to analyze signal from the high-speed train. In Fig. 1 is presented a sample explanation of our research assumptions. We assume that high-speed train is moving at regular speed on the track. During motion vibration signal is read from suspension sensors and processed by developed LSTM-RNN to decide if the train is stable as visible in Fig. 2. If the system discovers malfunction or difference from regular situation on the track, it sends warning to the operator. As a result, driving of such high-speed train becomes more safe and easier.

After conducting experiments to get the best possible Accuracy of our predictor, we have used NAdam optimization algorithm. Because of high performance and short training times, it is widely used in machine learning research. To improve model Accuracy and reduce training even more, we have used learning rate decay. It allowed deep neural network to first quickly adapt to the given problem and then slowly polish the final efficiency. NAdam formula can be described as follows. First we need to compute values \(a_i\) and \(b_i\) which are later used for computing correlations \(\hat{a}_i\) and \(\hat{b}_i\):where \(\theta _1\) and \(\theta _2\) are constant hyper-parameters and z is a current gradient value of the error function. Correlations are described as follows:After that the formula for updating weights is defined as follows:where \(\gamma\) is a small, constant value and \(\eta\) is a learning rate. Next we apply NAG formula to classical Adam using equations below:Finally, we modify Adam algorithm update rule; thus, we need to change equations for \(\hat{a}_i\) and \(x_i\):We have used Mean Squared Logarithmic Error as fitness function for our model, since it gave well understanding of the training process for this type of data.

Specification of our computer is as follows:Because in our work we are dealing with time series data, we have chosen empirically to use Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM) neurons. However, to ensure that our choice was correct we have done a comparison between RNN-LSTM, classical artificial neural network (ANN) and other types of RNN models. Results are shown in Table 1. We can see that in our tests models which use Long Short-Term Memory neurons are having the best Accuracy. Our proposed model is giving result similar to Bi-LSTM; however, from our experiments we see that training of our model was much shorter, what gave us a big advance and justified our choice to present this idea as a main result of our research in this project.

In Fig. 11, we see comparison between original sensor readings and results of prediction from our developed deep learning system. When we analyze results, we can see that original readings without prediction and dumping system show higher fluctuations in each interval than our proposed deep learning model. As a result, we see that application of neural network is very efficient in analysis of data samples. For our experiments, we have selected RNN with LSTM as initial test has shown better results than for standard architecture. We have selected NAdam training model as the most efficient training algorithm in our research case. This selection was based on comparative analysis, which results were discussed in section of experiments. We have compared various error margins and prediction intervals. Results have shown that our model is efficient in case of error at level above 0.1 which is very good results. As a summary of our research experiments, we can conclude that proposed deep learning architecture is interesting design for data collections with sensor readings in time intervals.

In Table 5, we can see comparisons to other models. Our proposed model is working very well. Proposed LSTM-RNN has reached lowest value MSE among compared solutions. For compared MAE, proposed model is second among all compared. Therefore, we can see that proposed model is working very well with time series data. Our model is based on RNN with LSTM neurons, while other is using attention mechanisms or random sampling ideas. Results show that proposed by us simplified processing gives MSE and MAE lowest than other presented models. These results confirm efficiency of our idea. We have also tested other configurations in our experiments. After some simple testing with changing the layers type from ReLU to ELU, we have experienced no gain in Accuracy (in some cases even up to 1% decrease), what’s more the training time performance was highly decreased coming about 1/4 times longer than the ReLU. The reason is because of the higher computational complexity of ELU function and adding more negative data which are not only unusable for this model as all values should be above zero, but also creates additional noise. Therefore, we decided to present the proposed architecture as our final model in this task. Our proposed model has three sensitive points. One of them is time as we have to assume how many individual sensor readings we can take, so that both proposed neural network can properly predict and the relevant systems can react. The self-learning method may turn out to be another problem, i.e., when excessively monotonous sections of the route may influence neural network. The last vulnerable point is reality itself, as long as we can very accurately predict standard train runs, we cannot be sure whether they will always be like that, as heavy construction works near the traction can affect the entire system.

In our article, we proposed vibration prediction model by the use of LSTM-RNN deep learning architecture. We collaterally presented optional applications of those predictions to create a system allowing to decrease sensible vibrations to satisfying extent. As the entire architecture was developed based on the data from telegraphic sensors during a drive, the model was devised in regard to that sort of transport. However, being obtained with accurate measures and equipment, we will be able to recreate the systems on the rules of a different transport, including air transport, road transport and water transport. In the future, we are willing to focus on adaptive ability, which will yield in further development in deep learning models for smart transport.