nach oben

Neural Computing and Applications

Erschienen in:

22.07.2016 | Original Article

Time series forecasting by recurrent product unit neural networks

verfasst von: F. Fernández-Navarro, Maria Angeles de la Cruz, P. A. Gutiérrez, A. Castaño, C. Hervás-Martínez

Erschienen in: Neural Computing and Applications | Ausgabe 3/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Time series forecasting (TSF) consists on estimating models to predict future values based on previously observed values of time series, and it can be applied to solve many real-world problems. TSF has been traditionally tackled by considering autoregressive neural networks (ARNNs) or recurrent neural networks (RNNs), where hidden nodes are usually configured using additive activation functions, such as sigmoidal functions. ARNNs are based on a short-term memory of the time series in the form of lagged time series values used as inputs, while RNNs include a long-term memory structure. The objective of this paper is twofold. First, it explores the potential of multiplicative nodes for ARNNs, by considering product unit (PU) activation functions, motivated by the fact that PUs are specially useful for modelling highly correlated features, such as the lagged time series values used as inputs for ARNNs. Second, it proposes a new hybrid RNN model based on PUs, by estimating the PU outputs from the combination of a long-term reservoir and the short-term lagged time series values. A complete set of experiments with 29 data sets shows competitive performance for both model proposals, and a set of statistical tests confirms that they achieve the state of the art in TSF, with specially promising results for the proposed hybrid RNN. The experiments in this paper show that the recurrent model is very competitive for relatively large time series, where longer forecast horizons are required, while the autoregressive model is a good selection if the data set is small or if a low computational cost is needed.

Vorheriger Artikel A decision system using fuzzy cognitive map and multi-group data envelopment analysis to estimate hospitals’ outputs level

Nächster Artikel MHD flow of radiative micropolar nanofluid in a porous channel: optimal and numerical solutions

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

For the sake of clarity, reservoir representation is simplified: there is a link between each reservoir node and each PU, and all reservoir nodes receive \(y_{t-1}\) time series value as input. The interconnections between reservoir nodes are random. Internal connections of the reservoir are given by \({\varvec{\upkappa }}\).

Available at http://www.neural-forecasting-competition.com.

Which can be found at http://sci2s.ugr.es/keel/timeseries.php.

Scaling the input data to positive values is required to avoid having complex numbers as output of the basis function. Additionally, the scaling considered also avoids having inputs equal to zero or one.

http://www.uco.es/ayrna/datasets/MAPEresultsNCAA.

In these kind of problems small variations in the inputs could produce large changes in the output of the TS. This situation could be modelled with the product units basis functions (as they are potential basis functions).

Furquim G, Pessin G, Faiçal B, Mendiondo E, Ueyama J (2015) Improving the accuracy of a flood forecasting model by means of machine learning and chaos theory: a case study involving a real wireless sensor network deployment in brazil. In: Neural computing and applications, pp 1–13. doi:10.1007/s00521-015-1930-z

Arroyo J, Maté C (2009) Forecasting histogram time series with \(k\)-nearest neighbours methods. Int J Forecast 25(1):192–207CrossRef

Arriandiaga A, Portillo E, Sánchez J, Cabanes I, Pombo I (2015) A new approach for dynamic modelling of energy consumption in the grinding process using recurrent neural networks. In: Neural computing and applications, pp 1–16. doi:10.1007/s00521-015-1957-1

Hansen J, Nelson R (1997) Neural networks and traditional time series methods: a synergistic combination in state economic forecasts. IEEE Trans Neural Netw 8(4):863–873CrossRef

Sitte R, Sitte J (2000) Analysis of the predictive ability of time delay neural networks applied to the S&P 500 time series. IEEE Trans Syst Man Cybern Part C Appl Rev 30(4):568–572CrossRef

Connor J, Martin R, Atlas L (1994) Recurrent neural networks and robust time series prediction. IEEE Trans Neural Netw 5(2):240–254CrossRef

He X, Li C, Huang T, Li C, Huang J (2014) A recurrent neural network for solving bilevel linear programming problem. IEEE Trans Neural Netw Learn Syst 25(4):824–830CrossRef

Yan Z, Wang J (2014) Robust model predictive control of nonlinear systems with unmodeled dynamics and bounded uncertainties based on neural networks. IEEE Trans Neural Netw Learn Syst 25(3):457–469MathSciNetCrossRef

Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780CrossRef

10.

Jaeger H (2002) Adaptive nonlinear system identification with echo state networks. In: Advances in neural information processing systems, pp 593–600

11.

Gallicchio C, Micheli A (2011) Architectural and markovian factors of echo state networks. Neural Netw 24(5):440–456CrossRef

12.

Rodan A, Tino P (2011) Minimum complexity echo state network. IEEE Trans Neural Netw 22(1):131–144CrossRef

13.

Hecht-Nielsen R (1989) Theory of the backpropagation neural network. In: International joint conference on neural networks 1989 IJCNN. IEEE, pp 593–605

14.

Pan F, Zhang H, Xia M (2009) A hybrid time-series forecasting model using extreme learning machines. In: Second international conference on Intelligent Computation Technology and Automation, ICICTA ’09, vol 1, pp 933–936

15.

Huang G-B, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B Cybern 42(2):513–529CrossRef

16.

Durbin R, Rumelhart D (1989) Products units: a computationally powerful and biologically plausible extension to backpropagation networks. Neural Comput 1(1):133–142CrossRef

17.

Goldberg DE et al (1989) Genetic algorithms in search, optimization, and machine learning, vol 412. Addison-Wesley, Reading Menlo ParkMATH

18.

Li P, Tan Z, Yan L, Deng K (2011) Time series prediction of mining subsidence based on genetic algorithm neural network. In: 2011 international symposium on computer science and society (ISCCS), pp 83–86

19.

Luque C, Ferran J, Vinuela P (2007) Time series forecasting by means of evolutionary algorithms. In: IEEE international 2007 parallel and distributed processing symposium, IPDPS 2007, pp 1–7

20.

Cai X, Zhang N, Venayagamoorthy G, Wunsch D (2004) Time series prediction with recurrent neural networks using a hybrid PSO–EA algorithm. In: 2004 IEEE international joint conference on neural networks, 2004. Proceedings, vol 2, pp 1647–1652

21.

Martínez-Estudillo FJ, Hervás-Martínez C, Gutiérrez PA, Martínez-Estudillo AC (2008) Evolutionary product-unit neural networks classifiers. Neurocomputing 72(1–3):548–561CrossRefMATH

22.

Martínez-Estudillo AC, Martínez-Estudillo FJ, Hervás-Martínez C, García-Pedrajas N (2006) Evolutionary product unit based neural networks for regression. Neural Netw 19(4):477–486CrossRefMATH

23.

Dulakshi AWJ, Karunasingha SK, Li WK (2011) Evolutionary product unit based neural networks for hydrological time series analysis. J Hydroinf 13(4):825–841CrossRef

24.

Piotrowski AP, Napiorkowski JJ (2012) Product-units neural networks for catchment runoff forecasting. Adv Water Resour 49:97–113CrossRef

25.

Sundermeyer M, Oparin I, Gauvain J-L, Freiberg B, Schluter R, Ney H (2013) Comparison of feedforward and recurrent neural network language models. In: 2013 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, pp 8430–8434

26.

Lukoševičius M, Jaeger H (2009) Reservoir computing approaches to recurrent neural network training. Comput Sci Rev 3(3):127–149CrossRefMATH

27.

Hansen N (2006) The CMA evolution strategy: a comparing review. In: Towards a new evolutionary computation. Studies in fuzziness and soft computing, vol 192. Springer, Berlin, pp 75–102

28.

Jastrebski G, Arnold D (2006) Improving evolution strategies through active covariance matrix adaptation. In: IEEE congress on 2006 evolutionary computation, CEC 2006, pp 2814–2821

29.

Heidrich-Meisner V, Igel C (2009) Neuroevolution strategies for episodic reinforcement learning. J Algorithms 64(4):152–168CrossRefMATH

30.

Moriguchi H, Honiden S (2012) CMA-TWEANN: efficient optimization of neural networks via self-adaptation and seamless augmentation. In: Proceedings of the 14th annual conference on genetic and evolutionary computation. ACM, pp 903–910

31.

Gundogdu O, Egrioglu E, Aladag C, Yolcu U (2015) Multiplicative neuron model artificial neural network based on gaussian activation function. Neural Comput Appl 27:927–935CrossRef

32.

Yadav R, Kalra P, John J (2007) Time series prediction with single multiplicative neuron model. Appl Soft Comput 7(4):1157–1163 (Soft computing for time series prediction)CrossRef

33.

Zhao L, Yang Y (2009) PSO-based single multiplicative neuron model for time series prediction. Expert Syst Appl 36(2):2805–2812 (Part 2)MathSciNetCrossRef

34.

Attia M, Sallam E, Fahmy M (Aug 2012) A proposed generalized mean single multiplicative neuron model. In: 2012 IEEE international conference on intelligent computer communication and processing (ICCP), pp 73–78

35.

Egrioglu E, Yolcu U, Aladag C, Bas E (2015) Recurrent multiplicative neuron model artificial neural network for non-linear time series forecasting. Neural Process Lett 41(2):249–258CrossRef

36.

Gutiérrez PA, Segovia-Vargas MJ, Salcedo-Sanz S, Hervás-Martínez C, Sanchís A, Portilla-Figueras JA, Fernández-Navarro F (2010) Hybridizing logistic regression with product unit and rbf networks for accurate detection and prediction of banking crises. Omega 38(5):333–344CrossRef

37.

Saini L, Soni M (2002) Artificial neural network based peak load forecasting using Levenberg–Marquardt and quasi-Newton methods. IEE Proc Gener Transm Distrib 149(5):578–584CrossRef

38.

Hansen N, Niederberger ASP, Guzzella L, Koumoutsakos P (2009) A method for handling uncertainty in evolutionary optimization with an application to feedback control of combustion. IEEE Trans Evolut Comput 13(1):180–197CrossRef

39.

Ros R, Hansen N (2008) A simple modification in CMA-ES achieving linear time and space complexity. In: Proceedings of the 10th international conference on parallel problem solving from nature: PPSN X. Springer, pp 296–305

40.

Huang G-B, Zhu Q-Y, Siew C-K (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: 2004 IEEE international joint conference on neural networks, 2004. Proceedings, vol 2, pp 985–990

41.

Ozturk MC, Xu D, Príncipe JC (2007) Analysis and design of echo state networks. Neural Comput 19(1):111–138CrossRefMATH

42.

Bergmeir C, Triguero I, Molina D, Aznarte J, Benitez J (2012) Time series modeling and forecasting using memetic algorithms for regimen-switching models. IEEE Trans Neural Netw Learn Syst 23(11):1841–1847CrossRef

43.

Alcalá-Fdez J, Fernández A, Luengo J, Derrac J, García S (2011) Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. Mult Valued Logic Soft Comput 17(2–3):255–287

44.

Said SE, Dickey DA (1984) Testing for unit roots in autoregressive-moving average models of unknown order. Biometrika 71(3):599–607MathSciNetCrossRefMATH

45.

Ragulskis M, Lukoseviciute K (2009) Non-uniform attractor embedding for time series forecasting by fuzzy inference systems. Neurocomputing 72(10):2618–2626CrossRef

46.

Crone S, Dhawan R (2007) Forecasting seasonal time series with neural networks: a sensitivity analysis of architecture parameters. In: International joint conference on neural networks, IJCNN 2007, pp 2099–2104

47.

Chow TWS, Leung C (1996) Nonlinear autoregressive integrated neural network model for short-term load forecasting. IEE Proc Gener Transm Distrib 143(5):500–506CrossRef

48.

Redel-Macías MD, Fernández-Navarro F, Gutiérrez PA, Cubero-Atienza AJ, Hervás-Martínez C (2013) Ensembles of evolutionary product unit or RBF neural networks for the identification of sound for pass-by noise test in vehicles. Neurocomputing 109:56–65CrossRef

49.

Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30MathSciNetMATH

50.

Zar JH et al (1999) Biostatistical analysis: Pearson Education India. Prentice Hall City, New Jersey

51.

Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 11(1):86–92MathSciNetCrossRefMATH

52.

Dunn OJ (1961) Multiple comparisons among means. J Am Stat Assoc 56(293):52–64MathSciNetCrossRefMATH

53.

Hochberg Y, Tamhane AC (1987) Multiple comparison procedures. Wiley, New YorkCrossRefMATH

54.

Aznarte JL, Alcalá-Fdez J, Arauzo-Azofra A, Benítez JM (2012) Financial time series forecasting with a bio-inspired fuzzy model. Expert Syst Appl 39(16):12302–12309CrossRef

55.

Adhikari R, Agrawal R (2012) Forecasting strong seasonal time series with artificial neural networks. J Sci Ind Res 71(10):657

56.

Rocha T, Paredes S, de Carvalho P, Henriques J (2013) An effective wavelet strategy for the trend prediction of physiological time series with application to phealth systems. In: 35th annual international conference of the IEEE engineering in medicine and biology society (EMBC) (2013). IEEE, pp 6788–6791

57.

Yao X (1999) Evolving artificial neural networks. Proc IEEE 87(9):1423–1447CrossRef

58.

Nikolaou A, Gutiérrez PA, Durán A, Dicaire I, Fernández-Navarro F, Hervás-Martínez C (2015) Detection of early warning signals in paleoclimate data using a genetic time series segmentation algorithm. Clim Dyn 44(7–8):1919–1933CrossRef

59.

Fernández-Navarro F, Hervás-Martínez C, Gutiérrez PA (2013) Generalised gaussian radial basis function neural networks. Soft Comput 17(3):519–533CrossRef

Titel: Time series forecasting by recurrent product unit neural networks
verfasst von: F. Fernández-Navarro
Maria Angeles de la Cruz
P. A. Gutiérrez
A. Castaño
C. Hervás-Martínez
Publikationsdatum: 22.07.2016
Verlag: Springer London
Erschienen in: Neural Computing and Applications / Ausgabe 3/2018
Print ISSN: 0941-0643
Elektronische ISSN: 1433-3058
DOI: https://doi.org/10.1007/s00521-016-2494-2

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Weitere Artikel der Ausgabe 3/2018

Similarity measures of bipolar neutrosophic sets and their application to multiple criteria decision making

Symbiotic organisms search optimization algorithm for economic/emission dispatch problem in power systems

A computational approach for nuclear export signals identification using spiking neural P systems

RBF-NN-based model for prediction of weld bead geometry in Shielded Metal Arc Welding (SMAW)

Machine aided malaria parasitemia detection in Giemsa-stained thin blood smears

Image salient regions encryption for generating visually meaningful ciphertext image

Premium Partner