Skip to main content
Erschienen in: Engineering with Computers 2/2022

07.01.2021 | Original Article

Solving the stochastic support vector regression with probabilistic constraints by a high-performance neural network model

verfasst von: Amir Feizi, Alireza Nazemi, Mohammad Reza Rabiei

Erschienen in: Engineering with Computers | Sonderheft 2/2022

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper offers a recurrent neural network to support vector machine (SVM) learning in stochastic support vector regression with probabilistic constraints. The SVM is first converted into an equivalent quadratic programming (QP) formulation in linear and nonlinear cases. An artificial neural network for SVM learning is then proposed. The presented neural network framework guarantees obtaining the optimal solution of the SVM problem. The existence and convergence of the trajectories of the network are studied. The Lyapunov stability for the considered neural network is also shown. The efficiency of the proposed method is shown by three illustrative examples.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Abaszade M, Effati S (2018) Stochastic support vector regression with probabilistic constraints. Appl Intell 48(1):243–256MATHCrossRef Abaszade M, Effati S (2018) Stochastic support vector regression with probabilistic constraints. Appl Intell 48(1):243–256MATHCrossRef
2.
Zurück zum Zitat Anguita D, Boni A (2002) Improved neural network for SVM learning. IEEE Trans Neural Netw 13(5):1243–1244CrossRef Anguita D, Boni A (2002) Improved neural network for SVM learning. IEEE Trans Neural Netw 13(5):1243–1244CrossRef
3.
Zurück zum Zitat Barnett NS, Dragomir SS, Agarwal R (2002) Some inequalities for probability, expectation, and variance of random variables defined over a finite interval. Comput Math Appl 43(10–11):1319–1357MathSciNetMATHCrossRef Barnett NS, Dragomir SS, Agarwal R (2002) Some inequalities for probability, expectation, and variance of random variables defined over a finite interval. Comput Math Appl 43(10–11):1319–1357MathSciNetMATHCrossRef
5.
Zurück zum Zitat Bazaraa MS, Sherali HD, Shetty CM (2013) Nonlinear programming: theory and algorithms. John Wiley & Sons Bazaraa MS, Sherali HD, Shetty CM (2013) Nonlinear programming: theory and algorithms. John Wiley & Sons
6.
Zurück zum Zitat Ben-Tal A, Bhadra S, Bhattacharyya C, Nath JS (2011) Chance constrained uncertain classification via robust optimization. Math Progr 127(1):145–173MathSciNetMATHCrossRef Ben-Tal A, Bhadra S, Bhattacharyya C, Nath JS (2011) Chance constrained uncertain classification via robust optimization. Math Progr 127(1):145–173MathSciNetMATHCrossRef
7.
Zurück zum Zitat Bennett KP, Mangasarian OL (1992) Robust linear programming discrimination of two linearly inseparable sets. Optim Methods Softw 1(1):23–34CrossRef Bennett KP, Mangasarian OL (1992) Robust linear programming discrimination of two linearly inseparable sets. Optim Methods Softw 1(1):23–34CrossRef
8.
Zurück zum Zitat Bhattacharyya C, Shivaswamy PK, Smola AJ (2005) A second order cone programming formulation for classifying missing data. In: Advances in neural information processing systems. pp 153–160 Bhattacharyya C, Shivaswamy PK, Smola AJ (2005) A second order cone programming formulation for classifying missing data. In: Advances in neural information processing systems. pp 153–160
9.
Zurück zum Zitat Boyd S, Vandenberghe L, Press CU (2004) Convex optimization. No. pt 1 in Berichte über verteilte messysteme. Cambridge University Press, Cambridge Boyd S, Vandenberghe L, Press CU (2004) Convex optimization. No. pt 1 in Berichte über verteilte messysteme. Cambridge University Press, Cambridge
10.
Zurück zum Zitat Carrizosa E, Gordillo J, Plastria F (2008) Kernel support vector regression with imprecise output. Dept. MOSI, Vrije Univ. Brussel, Brussel, Belgium, Tech. Rep Carrizosa E, Gordillo J, Plastria F (2008) Kernel support vector regression with imprecise output. Dept. MOSI, Vrije Univ. Brussel, Brussel, Belgium, Tech. Rep
11.
Zurück zum Zitat Chuang CC, Su SF, Jeng JT, Hsiao CC (2002) Robust support vector regression networks for function approximation with outliers. IEEE Trans Neural Netw 13(6):1322–1330CrossRef Chuang CC, Su SF, Jeng JT, Hsiao CC (2002) Robust support vector regression networks for function approximation with outliers. IEEE Trans Neural Netw 13(6):1322–1330CrossRef
12.
13.
Zurück zum Zitat Dierckx P (1995) Curve and surface fitting with splines. Oxford University Press Dierckx P (1995) Curve and surface fitting with splines. Oxford University Press
14.
Zurück zum Zitat Drucker H, Burges CJC, Kaufman L, Smola AJ, Vapnik V (1997) Support vector regression machines. Advances in neural information processing systems 9. MIT Press, Cambridge, pp 155–161 Drucker H, Burges CJC, Kaufman L, Smola AJ, Vapnik V (1997) Support vector regression machines. Advances in neural information processing systems 9. MIT Press, Cambridge, pp 155–161
15.
Zurück zum Zitat Farag AA, Mohamed RM (2003) Classification of multispectral data using support vector machines approach for density estimation. In: International conference on intelligent engineering system, pp 6–8. Citeseer Farag AA, Mohamed RM (2003) Classification of multispectral data using support vector machines approach for density estimation. In: International conference on intelligent engineering system, pp 6–8. Citeseer
16.
Zurück zum Zitat Gao X, Liao L (2009) A new projection-based neural network for constrained variational inequalities. IEEE Trans Neural Netw 20(3):373–388CrossRef Gao X, Liao L (2009) A new projection-based neural network for constrained variational inequalities. IEEE Trans Neural Netw 20(3):373–388CrossRef
17.
Zurück zum Zitat Hao PY (2010) New support vector algorithms with parametric insensitive/margin model. Neural Netw 23(1):60–73MATHCrossRef Hao PY (2010) New support vector algorithms with parametric insensitive/margin model. Neural Netw 23(1):60–73MATHCrossRef
20.
Zurück zum Zitat Hopfield J, Tank D (1986) Computing with neural circuits: a model. Science 233(4764):625–633CrossRef Hopfield J, Tank D (1986) Computing with neural circuits: a model. Science 233(4764):625–633CrossRef
21.
Zurück zum Zitat Hu X, Sun C, Zhang B (2010) Design of recurrent neural networks for solving constrained least absolute deviation problems. IEEE Trans Neural Netw 21(7):1073–1086CrossRef Hu X, Sun C, Zhang B (2010) Design of recurrent neural networks for solving constrained least absolute deviation problems. IEEE Trans Neural Netw 21(7):1073–1086CrossRef
22.
Zurück zum Zitat Hu X, Zhang B (2009) An alternative recurrent neural network for solving variational inequalities and related optimization problems. IEEE Trans Syst Man Cybern Part B (Cybern) 39(6):1640–1645CrossRef Hu X, Zhang B (2009) An alternative recurrent neural network for solving variational inequalities and related optimization problems. IEEE Trans Syst Man Cybern Part B (Cybern) 39(6):1640–1645CrossRef
23.
Zurück zum Zitat Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501CrossRef Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501CrossRef
24.
Zurück zum Zitat Kennedy MP, Chua LO (1988) Neural networks for nonlinear programming. IEEE Trans Circuits Syst 35(5):554–562MathSciNetCrossRef Kennedy MP, Chua LO (1988) Neural networks for nonlinear programming. IEEE Trans Circuits Syst 35(5):554–562MathSciNetCrossRef
25.
Zurück zum Zitat Lillo WE, Loh MH, Hui S, Zak SH (1993) On solving constrained optimization problems with neural networks: a penalty method approach. IEEE Trans Neural Netw 4(6):931–940CrossRef Lillo WE, Loh MH, Hui S, Zak SH (1993) On solving constrained optimization problems with neural networks: a penalty method approach. IEEE Trans Neural Netw 4(6):931–940CrossRef
26.
Zurück zum Zitat Fu Lin C, de Wang S (2004) Training algorithms for fuzzy support vector machines with noisy data. Pattern Recognit Lett 25(14):1647–1656CrossRef Fu Lin C, de Wang S (2004) Training algorithms for fuzzy support vector machines with noisy data. Pattern Recognit Lett 25(14):1647–1656CrossRef
28.
Zurück zum Zitat Lobo MS, Vandenberghe L, Boyd S, Lebret H (1998) Applications of second-order cone programming. Linear Algebra Appl 284(1):193–228MathSciNetMATHCrossRef Lobo MS, Vandenberghe L, Boyd S, Lebret H (1998) Applications of second-order cone programming. Linear Algebra Appl 284(1):193–228MathSciNetMATHCrossRef
30.
Zurück zum Zitat Miller R, Michel A (1982) Ordinary differential equations. Academic press, New YorkMATH Miller R, Michel A (1982) Ordinary differential equations. Academic press, New YorkMATH
31.
Zurück zum Zitat Nazemi A (2013) Solving general convex nonlinear optimization problems by an efficient neurodynamic model. Eng Appl Artif Intell 26(2):685–696CrossRef Nazemi A (2013) Solving general convex nonlinear optimization problems by an efficient neurodynamic model. Eng Appl Artif Intell 26(2):685–696CrossRef
32.
Zurück zum Zitat Nazemi A (2014) A neural network model for solving convex quadratic programming problems with some applications. Eng Appl Artif Intell 32:54–62CrossRef Nazemi A (2014) A neural network model for solving convex quadratic programming problems with some applications. Eng Appl Artif Intell 32:54–62CrossRef
33.
Zurück zum Zitat Nazemi A (2018) A capable neural network framework for solving degenerate quadratic optimization problems with an application in image fusion. Neural Process Lett 47(1):167–192CrossRef Nazemi A (2018) A capable neural network framework for solving degenerate quadratic optimization problems with an application in image fusion. Neural Process Lett 47(1):167–192CrossRef
34.
Zurück zum Zitat Nazemi A, Nazemi M (2014) A gradient-based neural network method for solving strictly convex quadratic programming problems. Cognit Comput 6(3):484–495MathSciNetCrossRef Nazemi A, Nazemi M (2014) A gradient-based neural network method for solving strictly convex quadratic programming problems. Cognit Comput 6(3):484–495MathSciNetCrossRef
35.
Zurück zum Zitat Peng X (2010) TSVR: an efficient twin support vector machine for regression. Neural Netw 23(3):365–372MATHCrossRef Peng X (2010) TSVR: an efficient twin support vector machine for regression. Neural Netw 23(3):365–372MATHCrossRef
36.
Zurück zum Zitat Perfetti R, Ricci E (2006) Analog neural network for support vector machine learning. IEEE Trans Neural Netw 17(4):1085–1091CrossRef Perfetti R, Ricci E (2006) Analog neural network for support vector machine learning. IEEE Trans Neural Netw 17(4):1085–1091CrossRef
37.
Zurück zum Zitat Sankowski D (2003) Signal processing systems. Theory and design. N. Kalouptsidis, a Wiley-interscience Publication, New York, 1997. Int J Adapt Control Signal Process 17(3):262–263CrossRef Sankowski D (2003) Signal processing systems. Theory and design. N. Kalouptsidis, a Wiley-interscience Publication, New York, 1997. Int J Adapt Control Signal Process 17(3):262–263CrossRef
38.
Zurück zum Zitat Schölkopf B, Smola AJ, Williamson RC, Bartlett PL (2000) New support vector algorithms. Neural Comput 12(5):1207–1245CrossRef Schölkopf B, Smola AJ, Williamson RC, Bartlett PL (2000) New support vector algorithms. Neural Comput 12(5):1207–1245CrossRef
39.
Zurück zum Zitat Shivaswamy PK, Bhattacharyya C, Smola AJ (2006) Second order cone programming approaches for handling missing and uncertain data. J Mach Learn Res 7(47):1283–1314MathSciNetMATH Shivaswamy PK, Bhattacharyya C, Smola AJ (2006) Second order cone programming approaches for handling missing and uncertain data. J Mach Learn Res 7(47):1283–1314MathSciNetMATH
40.
41.
Zurück zum Zitat Tank D, Hopfield J (1986) Simple ’neural’ optimization networks: an A/D converter, signal decision circuit, and a linear programming circuit. IEEE Trans Circuits Syst 33(5):533–541CrossRef Tank D, Hopfield J (1986) Simple ’neural’ optimization networks: an A/D converter, signal decision circuit, and a linear programming circuit. IEEE Trans Circuits Syst 33(5):533–541CrossRef
42.
Zurück zum Zitat Trafalis TB, Alwazzi SA (2007) Support vector regression with noisy data: a second order cone programming approach. Int J Gen Syst 36(2):237–250MathSciNetMATHCrossRef Trafalis TB, Alwazzi SA (2007) Support vector regression with noisy data: a second order cone programming approach. Int J Gen Syst 36(2):237–250MathSciNetMATHCrossRef
43.
Zurück zum Zitat Trafalis TB, Gilbert RC (2006) Robust classification and regression using support vector machines. Eur J Oper Res 173(3):893–909MathSciNetMATHCrossRef Trafalis TB, Gilbert RC (2006) Robust classification and regression using support vector machines. Eur J Oper Res 173(3):893–909MathSciNetMATHCrossRef
44.
Zurück zum Zitat Vapnik V (2013) The nature of statistical learning theory. Information science and statistics. Springer, New York Vapnik V (2013) The nature of statistical learning theory. Information science and statistics. Springer, New York
45.
Zurück zum Zitat Vapnik V, Golowich SE, Smola AJ (1997) Support vector method for function approximation, regression estimation and signal processing. In: Advances in neural information processing systems, pp 281–287 Vapnik V, Golowich SE, Smola AJ (1997) Support vector method for function approximation, regression estimation and signal processing. In: Advances in neural information processing systems, pp 281–287
46.
Zurück zum Zitat Vapnik V, Mukherjee S (2000) Support vector method for multivariate density estimation. In: Advances in neural information processing systems, pp 659–665 Vapnik V, Mukherjee S (2000) Support vector method for multivariate density estimation. In: Advances in neural information processing systems, pp 659–665
47.
Zurück zum Zitat Yang Y, Cao J (2008) A feedback neural network for solving convex constraint optimization problems. Appl Math Comput 201(1):340–350MathSciNetMATH Yang Y, Cao J (2008) A feedback neural network for solving convex constraint optimization problems. Appl Math Comput 201(1):340–350MathSciNetMATH
48.
Zurück zum Zitat Yang Y, He Q, Hu X (2012) A compact neural network for training support vector machines. Neurocomputing 86:193–198CrossRef Yang Y, He Q, Hu X (2012) A compact neural network for training support vector machines. Neurocomputing 86:193–198CrossRef
49.
Zurück zum Zitat Tan Y, Xia Y, Wang J (2000) Neural network realization of support vector methods for pattern classification. In: Proceedings of the IEEE-INNS-ENNS international joint conference on neural networks. IJCNN 2000. Neural computing: new challenges and perspectives for the New Millennium, vol 6, pp 411–416 Tan Y, Xia Y, Wang J (2000) Neural network realization of support vector methods for pattern classification. In: Proceedings of the IEEE-INNS-ENNS international joint conference on neural networks. IJCNN 2000. Neural computing: new challenges and perspectives for the New Millennium, vol 6, pp 411–416
50.
Zurück zum Zitat Yoshikawa T (1990) Foundations of robotics: analysis and control. East European Monographs, vol 279. MIT Press, Cambridge Yoshikawa T (1990) Foundations of robotics: analysis and control. East European Monographs, vol 279. MIT Press, Cambridge
51.
Zurück zum Zitat Youshen X, Jun W (2004) A one-layer recurrent neural network for support vector machine learning. IEEE Trans Syst Man Cybern Part B (Cybern) 34(2):1261–1269CrossRef Youshen X, Jun W (2004) A one-layer recurrent neural network for support vector machine learning. IEEE Trans Syst Man Cybern Part B (Cybern) 34(2):1261–1269CrossRef
52.
Zurück zum Zitat Xia Y, Wang J (2005) A recurrent neural network for solving nonlinear convex programs subject to linear constraints. IEEE Trans Neural Netw 16(2):379–386CrossRef Xia Y, Wang J (2005) A recurrent neural network for solving nonlinear convex programs subject to linear constraints. IEEE Trans Neural Netw 16(2):379–386CrossRef
53.
Zurück zum Zitat Zhao Y, Liu Q (2012) Generalized recurrent neural network for \(\epsilon\)-insensitive support vector regression. Math Comput Simul 86:2–9MathSciNetMATHCrossRef Zhao Y, Liu Q (2012) Generalized recurrent neural network for \(\epsilon\)-insensitive support vector regression. Math Comput Simul 86:2–9MathSciNetMATHCrossRef
54.
Zurück zum Zitat Zhao YP, Zhao J, Zhao M (2013) Twin least squares support vector regression. Neurocomputing 118:225–236CrossRef Zhao YP, Zhao J, Zhao M (2013) Twin least squares support vector regression. Neurocomputing 118:225–236CrossRef
Metadaten
Titel
Solving the stochastic support vector regression with probabilistic constraints by a high-performance neural network model
verfasst von
Amir Feizi
Alireza Nazemi
Mohammad Reza Rabiei
Publikationsdatum
07.01.2021
Verlag
Springer London
Erschienen in
Engineering with Computers / Ausgabe Sonderheft 2/2022
Print ISSN: 0177-0667
Elektronische ISSN: 1435-5663
DOI
https://doi.org/10.1007/s00366-020-01214-5

Weitere Artikel der Sonderheft 2/2022

Engineering with Computers 2/2022 Zur Ausgabe

Neuer Inhalt