Abstract
Aiming to solve the problems of low accuracy of multi-step prediction and difficulty in determining the maximum number of prediction steps of chaotic time series, a multi-step time series prediction model based on the dilated convolution network and long short-term memory (LSTM), named the dilated convolution-long short-term memory (DC-LSTM), is proposed. The dilated convolution operation is used to extract the correlation between the predicted variable and correlational variables. The features extracted by dilated convolution operation and historical data of predicted variable are input into LSTM to obtain the desired multi-step prediction result. Furthermore, cross-correlation analyses (CCA) are applied to calculate the reasonable maximum prediction steps of chaotic time series. Actual applications of multi-step prediction were studied to demonstrate the effectiveness of the proposed model which has superiorities in RMSE, MAE and prediction accuracy because of the extraction of correlation between the predicted variable and correlational variables. Moreover, the proposed DC-LSTM model provides a new method for prediction of chaotic time series and lays a foundation for scientific data analysis of chaotic time series monitoring systems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ai S, Chakravorty A, Rong C (2019) Household power demand prediction using evolutionary ensemble neural network pool with multiple network structures. Sensors (Basel) 19(3):721. https://doi.org/10.3390/s19030721
Alexander J Jr, Edwards RA, Brodsky M, Manca L, Grugni R, Savoldelli A, Bonfanti G, Emir B, Whalen E, Watt S, Parsons B (2018) Using time series analysis approaches for improved prediction of pain outcomes in subgroups of patients with painful diabetic peripheral neuropathy. PLoS One 13(12):e0207120. https://doi.org/10.1371/journal.pone.0207120
Awad M, Qasrawi I (2018) Enhanced RBF neural network model for time series prediction of solar cells panel depending on climate conditions (temperature and irradiance). Neural Comput Appl 30(6):1757–1768. https://doi.org/10.1007/s00521-016-2779-5
Baniata LH, Park S, Park S-B (2018) A multitask-based neural machine translation model with part-of-speech tags integration for Arabic Dialects. Appl Sci (Basel) 8(12):2502. https://doi.org/10.3390/app8122502
Borovykh A, Bohte S, Oosterlee CW (2019) Dilated convolutional neural networks for time series forecasting. J Comput Financ 22(4):73–101. https://doi.org/10.21314/jcf.2018.358
Cai M, Pipattanasomporn M, Rahman S (2019) Day-ahead building-level load forecasts using deep learning vs. traditional time-series techniques. Appl Energy 236:1078–1088. https://doi.org/10.1016/j.apenergy.2018.12.042
Cao LY (1997) Practical method for determining the minimum embedding dimension of a scalar time series. Physica D 110(1–2):43–50. https://doi.org/10.1016/s0167-2789(97)00118-8
Cao J, Li Z, Li J (2019) Financial time series forecasting model based on CEEMDAN and LSTM. Phys A Stat Mech Appl 519:127–139. https://doi.org/10.1016/j.physa.2018.11.061
Chang S-Y, Li B, Simko G, Sainath TN, Tripathi A, van den Oord A, Vinyals O, IEEE (2018) Temporal modeling using dilated convolution and gating for voice-activity-detection. In: 2018 IEEE International conference on acoustics, speech and signal processing. https://doi.org/10.1109/icassp.2018.8461921
Chen WL, Yeo CK, Lau CT, Lee BS (2018) Leveraging social media news to predict stock index movement using RNN-boost. Data Knowl Eng 118:14–24. https://doi.org/10.1016/j.datak.2018.08.003
Cheng F, Li T, Wei Y-M, Fan T (2019) The VEC-NAR model for short-term forecasting of oil prices. Energy Econ 78:656–667. https://doi.org/10.1016/j.eneco.2017.12.035
Chlouverakis KE, Adams MJ (2003) Stability maps of injection-locked laser diodes using the largest Lyapunov exponent. Opt Commun 216(4–6):405–412. https://doi.org/10.1016/s0030-4018(02)02357-x
Cirstea R-G, Micu D-V, Muresan G-M, Guo C, Yang B (2018) Correlated time series forecasting using multi-task deep neural networks. In: Cikm’18: Proceedings of the 27th ACM international conference on information and knowledge management. https://doi.org/10.1145/3269206.3269310
Dhanya CT, Kumar DN (2010) Nonlinear ensemble prediction of chaotic daily rainfall. Adv Water Resour 33(3):327–347. https://doi.org/10.1016/j.advwatres.2010.01.001
Fraser AM, Swinney HL (1986) Independent coordinates for strange attractors from mutual information. Phys Rev A 33(2):1134–1140. https://doi.org/10.1103/PhysRevA.33.1134
Helwig N, Pignanelli E, Schutze A, IEEE (2015) Condition monitoring of a complex hydraulic system using multivariate statistics. In: 2015 IEEE International instrumentation and measurement technology conference. IEEE, New York. https://doi.org/10.1109/i2mtc.2015.7151267
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
Hua Y, Mou L, Zhu XX (2019) Recurrently exploring class-wise attention in a hybrid convolutional and bidirectional LSTM network for multi-label aerial image classification. ISPRS J Photogramm Remote Sens 149:188–199. https://doi.org/10.1016/j.isprsjprs.2019.01.015
Karunasinghe DSK, Liong SY (2006) Chaotic time series prediction with a global model: artificial neural network. J Hydrol 323(1–4):92–105. https://doi.org/10.1016/j.jhydrol.2005.07.048
Ke J, Zheng H, Yang H, Chen X (2017) Short-term forecasting of passenger demand under on-demand ride services: a spatio-temporal deep learning approach. Transp Res C Emerg Technol 85:591–608. https://doi.org/10.1016/j.trc.2017.10.016
Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324. https://doi.org/10.1109/5.726791
Li Y, Zhang X, Chen D, IEEE (2018) CSRNet: Dilated convolutional neural networks for understanding the highly congested scenes. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition. IEEE conference on computer vision and pattern recognition, pp 1091–1100. https://doi.org/10.1109/cvpr.2018.00120
Liang YJ, Gao ZY, Gao JM, Wang RX, Zhao H (2018) Data fusion combined with echo state network for multivariate time series prediction in complex electromechanical system. Comput Appl Math 37(5):5920–5934. https://doi.org/10.1007/s40314-018-0669-4
Liu J, Zhang T, Han G, Gou Y (2018) TD-LSTM: temporal dependence-based LSTM networks for marine temperature prediction. Sensors (Basel) 18(11):89. https://doi.org/10.3390/s18113797
Liu A-A, Shao Z, Wong Y, Li J, Su Y-T, Kankanhalli M (2019) LSTM-based multi-label video event detection. Multimed Tools Appl 78(1):677–695. https://doi.org/10.1007/s11042-017-5532-x
Lopez E, Valle C, Allende H, Gil E, Madsen H (2018) Wind power forecasting based on echo state networks and long short-term memory. Energies 11(3):526. https://doi.org/10.3390/en11030526
Pal SS, Kar S (2017) Time series forecasting using fuzzy transformation and neural network with back propagation learning. J Intell Fuzzy Syst 33(1):467–477. https://doi.org/10.3233/jifs-161767
Pang N, Yin F, Zhang X, Zhao X (2017) A robust approach for multivariate time series forecasting. In: Proceedings of the eighth international symposium on information and communication technology. ACM. https://doi.org/10.1145/3155133.3155172
Pannakkong W, Sriboonchitta S, Huynh V-N (2018) An ensemble model of Arima and Ann with restricted boltzmann machine based on decomposition of discrete wavelet transform for time series forecasting. J Syst Sci Syst Eng 27(5):690–708. https://doi.org/10.1007/s11518-018-5390-8
Pano-Azucena AD, Tlelo-Cuautle E, Tan SXD, IEEE (2018) Prediction of chaotic time series by using ANNs, ANFIS and SVMs. In: 2018 7th International conference on modern circuits and systems technologies. IEEE, New York. https://doi.org/10.1109/mocast.2018.8376560
Perkel DH, Gerstein GL, Moore GP (1967) Neuronal spike trains and stochastic point processes. 2. Simultaneous spike trains. Biophys J 7(4):419. https://doi.org/10.1016/s0006-3495(67)86597-4
Rhodes C, Morari M (1997) The false nearest neighbors algorithm: an overview. Comput Chem Eng 21:S1149–S1154. https://doi.org/10.1016/s0098-1354(97)00204-4
Sivakumar B (2002) A phase-space reconstruction approach to prediction of suspended sediment concentration in rivers. J Hydrol 258(1–4):149–162. https://doi.org/10.1016/s0022-1694(01)00573-x
Strubell E, Verga P, Belanger D, McCallum A (2017) Fast and accurate entity recognition with iterated dilated convolutions. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 2670–2680. https://doi.org/10.18653/v1/d17-1283
Sun X, Dong K, Ma L, Sutcliffe R, He F, Chen S, Feng J (2019) Drug–drug interaction extraction via recurrent hybrid convolutional neural networks with an improved focal loss. Entropy (Switz) 21(1):37. https://doi.org/10.3390/e21010037
Sundermeyer M, Ney H, Schlueter R (2015) From feedforward to recurrent LSTM neural networks for language modeling. IEEE ACM Trans Audio Speech Lang Process 23(3):517–529. https://doi.org/10.1109/taslp.2015.2400218
Takens F (1981) Detecting strange attractors in turbulence. Dynamical systems and turbulence, Warwick 1980. Springer, Berlin, pp 366–381. https://doi.org/10.1007/bfb0091924
Tan K, Chen J, Wang D (2019) Gated residual networks with dilated convolutions for monaural speech enhancement. IEEE ACM Trans Audio Speech Lang 27(1):189–198. https://doi.org/10.1109/taslp.2018.2876171
Tian Y, Zhang K, Li J, Lin X, Yang B (2018) LSTM-based traffic flow prediction with missing data. Neurocomputing 318:297–305. https://doi.org/10.1016/j.neucom.2018.08.067
Vlachas PR, Byeon W, Wan ZY, Sapsis TP, Koumoutsakos P (2018) Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks. Proc R Soc Math Phys Eng Sci 474(2213):20170844. https://doi.org/10.1098/rspa.2017.0844
Wang J, Zhang C (2018) Software reliability prediction using a deep learning model based on the RNN encoder–decoder. Reliab Eng Syst Saf 170:73–82. https://doi.org/10.1016/j.ress.2017.10.019
Wang H, Yang W, Yuan C, Ling H, Hu W (2017a) Human activity prediction using temporally-weighted generalized time warping. Neurocomputing 225:139–147. https://doi.org/10.1016/j.neucom.2016.11.004
Wang R, Gao J, Gao Z, Gao X, Jiang H (2017b) Analysis of multifractality of multivariable coupling relationship of complex electromechanical system in process industry. Proc Inst Mech Eng Part E J Process Mech Eng 231(6):1087–1100. https://doi.org/10.1177/0954408916653149
Wang R, Gao J, Gao Z, Gao X, Jiang H, Liang Z (2017c) Interaction analysis-based information modeling of complex electromechanical systems in the processing industry. Proc Inst Mech Eng Part I J Syst Control Eng 231(8):638–651. https://doi.org/10.1177/0959651817718454
Wang R, Gao X, Gao J, Gao Z, Kang J (2018) An information transfer based novel framework for fault root cause tracing of complex electromechanical systems in the processing industry. Mech Syst Signal Process 101:121–139. https://doi.org/10.1016/j.ymssp.2017.08.030
Wang B, Lei Y, Tian S, Wang T, Liu Y, Patel P, Jani AB, Mao H, Curran WJ, Liu T, Yang X (2019) Deeply supervised 3D fully convolutional networks with group dilated convolution for automatic MRI prostate segmentation. Med Phys 46(4):1707–1718. https://doi.org/10.1002/mp.13416
Wiatowski T, Bölcskei H (2017) A mathematical theory of deep convolutional neural networks for feature extraction. IEEE Trans Inf Theory 64(3):1845–1866. https://doi.org/10.1109/TIT.2017.2776228
Wolf A, Swift JB, Swinney HL, Vastano JA (1985) Determining Lyapunov exponents from a time series. Physica D 16(3):285–317. https://doi.org/10.1016/0167-2789(85)90011-9
Xiao J, Zhu X, Huang C, Yang X, Wen F, Zhong M (2019) A new approach for stock price analysis and prediction based on SSA and SVM. Int J Inf Technol Decis 18(1):287–310. https://doi.org/10.1142/s021962201841002x
Yeh C-H, Lin M-H, Lin C-H, Yu C-E, Chen M-J (2019) Machine learning for long cycle maintenance prediction of wind turbine. Sensors (Basel, Switzerland) 19(7):1671. https://doi.org/10.3390/s19071671
Zhang H, Li H, Yang B, Lu J (2017) Process time series prediction based on application of correlated process variables to CNN time delayed analyses. CIESC J 68(9):3501–3510. https://doi.org/10.11949/j.issn.0438-1157.20170197
Zhang J, Wang P, Yan R, Gao RX (2018) Long short-term memory for machine remaining life prediction. J Manuf Syst 48:78–86. https://doi.org/10.1016/j.jmsy.2018.05.011
Zhao J, Mao X, Chen L (2019) Speech emotion recognition using deep 1D&2D CNN LSTM networks. Biomed Signal Process 47:312–323. https://doi.org/10.1016/j.bspc.2018.08.035
Zhou TL, Gao SC, Wang JH, Chu CY, Todo Y, Tang Z (2016) Financial time series prediction using a dendritic neuron model. Knowl Based Syst 105:214–224. https://doi.org/10.1016/j.knosys.2016.05.031
Zhou Y, Chang F-J, Chang L-C, Kao IF, Wang Y-S, Kang C-C (2019) Multi-output support vector machine for regional multi-step-ahead PM2.5 forecasting. Sci Total Environ 651:230–240. https://doi.org/10.1016/j.scitotenv.2018.09.111
Acknowledgements
All authors, especially the corresponding author Rongxi Wang, would like to thank the anonymous reviewers for their time and invaluable comments and suggestions on this paper.
Funding
This research was funded in part by the National Nature Science Foundation of China, Grant number 51905409, in part by the National Key R&D program of China, Grant number 2017YFF0210500, and in part by the China Postdoctoral Science Foundation project, Grant number 2017M620446.
Author information
Authors and Affiliations
Contributions
Formal analysis, RW and HJ; investigation, RW, CP, JG and ZG; methodology, RW; project administration, RW and JG; writing—original draft, RW; writing—review and editing, RW.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflict of interest.
Additional information
Communicated by Marcos Eduardo Valle.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wang, R., Peng, C., Gao, J. et al. A dilated convolution network-based LSTM model for multi-step prediction of chaotic time series. Comp. Appl. Math. 39, 30 (2020). https://doi.org/10.1007/s40314-019-1006-2
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s40314-019-1006-2