nach oben

Neural Computing and Applications

Erschienen in:

19.03.2016 | Original Article

SVM hyperparameters tuning for recursive multi-step-ahead prediction

verfasst von: Jie Liu, Enrico Zio

Erschienen in: Neural Computing and Applications | Ausgabe 12/2017

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Prediction of time series data is of relevance for many industrial applications. The prediction can be made in one-step and multi-step ahead. For predictive maintenance, multi-step-ahead prediction is of interest for projecting the evolution of the future conditions of the equipment of interest, computing the remaining useful life and taking corresponding maintenance decisions. Recursive prediction is one of the popular strategies for multi-step-ahead prediction. SVM is a popular data-driven approach that has been used for recursive multi-step-ahead prediction. Tuning the hyperparameters in SVM during the training process is challenging, and normally the hyperparameters are tuned by solving an optimization problem. This paper analyses the possible objectives of the optimization for tuning hyperparameters. Through experiments on one synthetic dataset and two real time series data, related to the prediction of wind speed in a region and leakage from the reactor coolant pump in a nuclear power plant, a bi-objective optimization combining mean absolute derivatives and accuracy on all prediction steps is shown to be the best choice for tuning SVM hyperparameters for recursive multi-step-ahead prediction.

Vorheriger Artikel A P300 potential evaluation wavelet method comparing individuals with high and low risk for alcoholism

Nächster Artikel Optimization of 5.5-GHz CMOS LNA parameters using firefly algorithm

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Aich U, Banerjee S (2013) Modeling of EDM responses by support vector machine regression with parameters selected by particle swarm optimization. Appl Math Model 38:2800–2818. doi:10.1016/j.apm.2013.10.073 CrossRef

Ak R, Li Y, Vitelli V, Zio E, López Droguett E, Magno Couto Jacinto C (2013) NSGA-II-trained neural network approach to the estimation of prediction intervals of scale deposition rate in oil & gas equipment. Expert Syst Appl 40:1205–1212. doi:10.1016/j.eswa.2012.08.018 CrossRef

Asefa T, Kemblowski M, McKee M, Khalil A (2006) Multi-time scale stream flow predictions: the support vector machines approach. J Hydrol 318:7–16. doi:10.1016/j.jhydrol.2005.06.001 CrossRef

Aydin I, Karakose M, Akin E (2011) A multi-objective artificial immune algorithm for parameter optimization in support vector machine. Appl Soft Comput 11:120–129. doi:10.1016/j.asoc.2009.11.003 CrossRef

Bao Y, Xiong T, Hu Z (2014) Multi-step-ahead time series prediction using multiple-output support vector regression. Neurocomputing 129:482–493. doi:10.1016/j.neucom.2013.09.010 CrossRef

Belegundu AD, Chandrupatla TR (2011) Optimization concepts and applications in engineering. Cambridge University Press

Benkedjouh T, Medjaher K, Zerhouni N, Rechak S (2013) Remaining useful life estimation based on nonlinear feature reduction and support vector regression. Eng Appl Artif Intell 26:1751–1760. doi:10.1016/j.engappai.2013.02.006 CrossRef

Benkedjouh T, Medjaher K, Zerhouni N, Rechak S (2015) Health assessment and life prediction of cutting tools based on support vector regression. J Intell Manuf 26:213–223. doi:10.1007/s10845-013-0774-6 CrossRef

Blasco X, Herrero JM, Sanchis J, Martínez M (2008) A new graphical visualization of n-dimensional Pareto front for decision-making in multiobjective optimization. Inf Sci 178:3908–3924. doi:10.1016/j.ins.2008.06.010 CrossRefMATH

10.

Canadian Weather Office (2012) The wind speed data. http://climate.weather.gc.ca/

11.

Chang C-C, Chou S-H (2015) Tuning of the hyperparameters for L2-loss SVMs with the RBF kernel by the maximum-margin principle and the jackknife technique. Pattern Recognit 48:3983–3992. doi:10.1016/j.patcog.2015.06.017 CrossRef

12.

Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46:131–159. doi:10.1023/A:1012450327387 CrossRefMATH

13.

Chen R, Liang C-Y, Hong W-C, Gu D-X (2015) Forecasting holiday daily tourist flow based on seasonal support vector regression with adaptive genetic algorithm. Appl Soft Comput 26:435–443. doi:10.1016/j.asoc.2014.10.022 CrossRef

14.

Chou J-SS, Cheng M-YY, Wu Y-WW, Pham A-DD (2014) Optimizing parameters of support vector machine using fast messy genetic algorithm for dispute classification. Expert Syst Appl 41:3955–3964. doi:10.1016/j.eswa.2013.12.035 CrossRef

15.

Ding S, Han Y, Yu J, Gu Y (2013) A fast fuzzy support vector machine based on information granulation. Neural Comput Appl 23(1):139–144CrossRef

16.

Drucker H, Burges CJC, Kaufman L et al (1997) Support vector regression machines. In: Mozer MC, Jordan MI, Petsche T (eds) Advances in neural information processing systems. vol 9. MIT Press, Cambridge, pp 155–161

17.

Hu Z, Bao Y, Xiong T (2014) Comprehensive learning particle swarm optimization based memetic algorithm for model selection in short-term load forecasting using support vector regression. Appl Soft Comput 25:15–25. doi:10.1016/j.asoc.2014.09.007 CrossRef

18.

Igel C (2005) Multi-objective model selection for support vector machines. In: Coello CAC, Aguirre AH, Zitzler E (eds) Evolutionary multi-criterion optimization. Springer, Berlin, pp 534–546

19.

Jardine AKS, Lin D, Banjevic D (2006) A review on machinery diagnostics and prognostics implementing condition-based maintenance. Mech Syst Signal Process 20:1483–1510. doi:10.1016/j.ymssp.2005.09.012 CrossRef

20.

Javed K, Gouriveau R, Zerhouni N, Nectoux P (2015) Enabling health monitoring approach based on vibration data for accurate prognostics. IEEE Trans Ind Electron 62:647–656. doi:10.1109/TIE.2014.2327917 CrossRef

21.

Karush W (1939) Minima of functions of several variables with inequalities as side constraints (doctoral dissertation, Master’s thesis, Dept. of Mathematics, Univ. of Chicago)

22.

Kim IY, De Weck OL (2005) Adaptive weighted-sum method for bi-objective optimization: Pareto front generation. Struct Multidiscip Optim 29:149–158. doi:10.1007/s00158-004-0465-1 CrossRef

23.

Kuhn HW, Tucker A (1951) Nonlinear programming. In: Proceedings of the second symposium on mathematical statistics and probability. doi:10.1007/BF01582292

24.

Lei Y, He Z, Zi Y, Hu Q (2007) Fault diagnosis of rotating machinery based on multiple ANFIS combination with GAs. Mech Syst Signal Process 21:2280–2294. doi:10.1016/j.ymssp.2006.11.003 CrossRef

25.

Lei Y, Liu Z, Wu X et al (2015) Health condition identification of multi-stage planetary gearboxes using a mRVM-based method. Mech Syst Signal Process 60–61:289–300. doi:10.1016/j.ymssp.2015.01.014 CrossRef

26.

Lin J-Y, Cheng C-T, Chau K-W (2006) Using support vector machines for long-term discharge prediction. Hydrol Sci J 51:599–612. doi:10.1623/hysj.51.4.599 CrossRef

27.

Liu R, Liu E, Yang J, Li M, Wang F (2006) Optimizing the hyper-parameters for SVM by combining evolution strategies with a grid search. Lect Notes Control Inf Sci 344:712–721. doi:10.1007/11816492_87 CrossRefMATH

28.

Liu S, Xu L, Jiang Y, Li D, Chen Y, Li Z (2014) A hybrid WA–CPSO-LSSVR model for dissolved oxygen content prediction in crab culture. Eng Appl Artif Intell 29:114–124. doi:10.1016/j.engappai.2013.09.019 CrossRef

29.

Min JH, Lee Y-C (2005) Bankruptcy prediction using support vector machine with optimal choice of kernel function parameters. Expert Syst Appl 28:603–614. doi:10.1016/j.eswa.2004.12.008 CrossRef

30.

Müller K, Smola A, Rätsch G, Schölkopf B, Kohlmorgen J, Vapnik V (1997) Predicting time series with support vector machines. In: Artificial neural networks ICANN97, pp 999–1004

31.

Namdari M, Jazayeri-Rad H (2014) Incipient fault diagnosis using support vector machines based on monitoring continuous decision functions. Eng Appl Artif Intell 28:22–35. doi:10.1016/j.engappai.2013.11.013 CrossRef

32.

Nuhic A, Terzimehic T, Soczka-Guth T, Buchholz M, Dietmayer K (2013) Health diagnosis and remaining useful life prognostics of lithium-ion batteries using data-driven methods. J Power Sources 239:680–688. doi:10.1016/j.jpowsour.2012.11.146 CrossRef

33.

Palm R (2007) Multiple-step-ahead prediction in control systems with Gaussian process models and TS-fuzzy models. Eng Appl Artif Intell 20:1023–1035. doi:10.1016/j.engappai.2007.02.003 CrossRef

34.

Rodgers JL, Nicewander WA (1988) Thirteen ways to look at the correlation coefficient. Am Stat 42:59–66. doi:10.2307/2685263 CrossRef

35.

Saha B, Goebel K, Poll S, Christophersen J (2009) Prognostics methods for battery health monitoring using a Bayesian framework. IEEE Trans Instrum Meas 58:291–297CrossRef

36.

Sapankevych N, Sankar R (2009) Time series prediction using support vector machines: a survey. IEEE Comput Intell Mag 4:24–38. doi:10.1109/MCI.2009.932254 CrossRef

37.

Shen L, Chen H, Yu Z et al (2016) Evolving support vector machines using fruit fly optimization for medical data classification. Knowl Based Syst. doi:10.1016/j.knosys.2016.01.002

38.

Sorjamaa A, Hao J, Reyhani N, Ji Y, Lendasse A (2007) Methodology for long-term prediction of time series. Neurocomputing 70:2861–2869. doi:10.1016/j.neucom.2006.06.015 CrossRef

39.

Taieb SB, Sorjamaa A, Bontempi G (2010) Multiple-output modeling for multi-step-ahead time series forecasting. Neurocomputing 73:1950–1957. doi:10.1016/j.neucom.2009.11.030 CrossRef

40.

Vichare NM, Pecht MG (2006) Prognostics and health management of electronics. IEEE Trans Compon Packag Technol 29:222–229. doi:10.1109/TCAPT.2006.870387 CrossRef

41.

Wang X, Han M (2015) Improved extreme learning machine for multivariate time series online sequential prediction. Eng Appl Artif Intell 40:28–36. doi:10.1016/j.engappai.2014.12.013 CrossRef

42.

Wang Z, He X, Gao D, Xue X (2013) An efficient Kernel-based matrixized least squares support vector machine. Neural Comput Appl 22(1):143–150CrossRef

43.

Wang S, Han Z, Liu F, Tang Y (2015) Nonlinear system identification using least squares support vector machine tuned by an adaptive particle swarm optimization. Int J Mach Learn Cybern 6:981–992. doi:10.1007/s13042-015-0403-0 CrossRef

44.

Widodo A, Shim MC, Caesarendra W, Yang BS (2011) Intelligent prognostics for battery health monitoring based on sample entropy. Expert Syst Appl 38:11763–11769. doi:10.1016/j.eswa.2011.03.063 CrossRef

45.

Wu CL, Chau KW (2013) Prediction of rainfall time series using modular soft computingmethods. Eng Appl Artif Intell 26:997–1007. doi:10.1016/j.engappai.2012.05.023 CrossRef

46.

Wu C-H, Tzeng G-H, Goo Y-J, Fang W-C (2007) A real-valued genetic algorithm to optimize the parameters of support vector machine for predicting bankruptcy. Expert Syst Appl 32:397–408. doi:10.1016/j.eswa.2005.12.008 CrossRef

47.

Zhang X, Qiu D, Chen F (2015) Support vector machine with parameter optimization by a novel hybrid method and its application to fault diagnosis. Neurocomputing 149:641–651. doi:10.1016/j.neucom.2014.08.010 CrossRef

48.

Zhao Z, Quan Q, Cai K-Y (2014) A profust reliability based approach to prognostics and health management. IEEE Trans Reliab 63:26–41. doi:10.1109/TR.2014.2299111 CrossRef

49.

Zio E (2012) Prognostics and health management of industrial equipment. Diagnostics and prognostics of engineering systems: methods and techniques, pp 333–356

50.

Zio E, Di Maio F (2012) Fatigue crack growth estimation by relevance vector machine. Expert Syst Appl 39:10681–10692. doi:10.1016/j.eswa.2012.02.199 CrossRef

Titel: SVM hyperparameters tuning for recursive multi-step-ahead prediction
verfasst von: Jie Liu
Enrico Zio
Publikationsdatum: 19.03.2016
Verlag: Springer London
Erschienen in: Neural Computing and Applications / Ausgabe 12/2017
Print ISSN: 0941-0643
Elektronische ISSN: 1433-3058
DOI: https://doi.org/10.1007/s00521-016-2272-1

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Weitere Artikel der Ausgabe 12/2017

High-order fuzzy-neuro-entropy integration-based expert system for time series forecasting

Evaluating different types of artificial neural network structures for performance prediction of compact heat exchanger

A new hybrid parametric and machine learning model with homogeneity hint for European-style index option pricing

Day-ahead forecasting of solar photovoltaic output power using multilayer perceptron

Deep learning in vision-based static hand gesture recognition

Integration of industrial videogrammetry and artificial neural networks for monitoring and modeling the deformation or displacement of structures

Premium Partner