Skip to main content
Erschienen in: Soft Computing 7/2015

01.07.2015 | Methodologies and Application

Incorporating prior knowledge and multi-kernel into linear programming support vector regression

verfasst von: Jinzhu Zhou, Baoyan Duan, Jin Huang, Na Li

Erschienen in: Soft Computing | Ausgabe 7/2015

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper proposes a multi-kernel linear program support vector regression with prior knowledge in order to obtain an accurate data-driven model in the case of an insufficient amount of measured data. In the algorithm, multiple feature spaces have been utilized to incorporate multi-kernel functions into the framework of linear programming support vector regression (LPSVR), and then the prior knowledge which may be exact or biased from a calibrated physical simulator has also been incorporated into LPSVR by modifying optimization formulations. Moreover, a strategy of parameter selections for the proposed algorithm has been presented to facilitate practical applications. Some experiments from a synthetic example, a microstrip antenna and six-pole microwave filter have been carried out, and the experimental results show that the proposed algorithm can obtain a satisfactory data-based model in the case of the scarcity of measured data. The proposed algorithm shows great potentialities in some applications where the experimental data are insufficient for an accurate data-driven model and the prior knowledge from a calibrated physical simulator of practical applications is available.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Bach FR, Lanckriet GRG, Jordan MI Multiple kernel learning, conic duality, and the SMO algorithm. In: Banff, Alta, Canada, 2004. Proceedings, Twenty-First International conference on machine learning, ICML 2004. Association for Computing Machinery, pp 41–48 Bach FR, Lanckriet GRG, Jordan MI Multiple kernel learning, conic duality, and the SMO algorithm. In: Banff, Alta, Canada, 2004. Proceedings, Twenty-First International conference on machine learning, ICML 2004. Association for Computing Machinery, pp 41–48
Zurück zum Zitat Bloch G, Lauer F, Colin G, Chamaillard Y (2008) Support vector regression from simulation data and few experimental samples. Inf Sci 178(20):3813–3827CrossRef Bloch G, Lauer F, Colin G, Chamaillard Y (2008) Support vector regression from simulation data and few experimental samples. Inf Sci 178(20):3813–3827CrossRef
Zurück zum Zitat Cherkassky V, Yunqian M (2004) Practical selection of SVM parameters and noise estimation for SVM regression. Neural Netw 17:113–126MATHCrossRef Cherkassky V, Yunqian M (2004) Practical selection of SVM parameters and noise estimation for SVM regression. Neural Netw 17:113–126MATHCrossRef
Zurück zum Zitat Clarke SM, Griebsch JH, Simpson TW (2005) Analysis of support vector regression for approximation of complex engineering analyses. Trans ASME J Mech Design 127(6):1077–1087CrossRef Clarke SM, Griebsch JH, Simpson TW (2005) Analysis of support vector regression for approximation of complex engineering analyses. Trans ASME J Mech Design 127(6):1077–1087CrossRef
Zurück zum Zitat Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines. Cambridge University Press, Cambridge Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines. Cambridge University Press, Cambridge
Zurück zum Zitat Huang CF (2012) A hybrid stock selection model using genetic algorithms and support vector regression. Appl Soft Comput 12(2):807–818CrossRef Huang CF (2012) A hybrid stock selection model using genetic algorithms and support vector regression. Appl Soft Comput 12(2):807–818CrossRef
Zurück zum Zitat Lanckriet GRG, Cristianini N, Bartlett P, El Ghaoui L, Jordan MI (2004) Learning the kernel matrix with semidefinite programming. J Mach Learn Res 5:27–72 Lanckriet GRG, Cristianini N, Bartlett P, El Ghaoui L, Jordan MI (2004) Learning the kernel matrix with semidefinite programming. J Mach Learn Res 5:27–72
Zurück zum Zitat Lauer F, Bloch G (2008a) Incorporating prior knowledge in support vector machines for classification: a review. Neurocomputing 71(7–9):1578–1594CrossRef Lauer F, Bloch G (2008a) Incorporating prior knowledge in support vector machines for classification: a review. Neurocomputing 71(7–9):1578–1594CrossRef
Zurück zum Zitat Lauer F, Bloch G (2008b) Incorporating prior knowledge in support vector regression. Mach Learn 70(1):89–118CrossRef Lauer F, Bloch G (2008b) Incorporating prior knowledge in support vector regression. Mach Learn 70(1):89–118CrossRef
Zurück zum Zitat Lu Z, Sun J (2009) Non-Mercer hybrid kernel for linear programming support vector regression in nonlinear systems identification. Appl Soft Comput 9(1):94–99CrossRef Lu Z, Sun J (2009) Non-Mercer hybrid kernel for linear programming support vector regression in nonlinear systems identification. Appl Soft Comput 9(1):94–99CrossRef
Zurück zum Zitat Lu Z, Sun J, Butts KR (2009) Linear programming support vector regression with wavelet kernel: a new approach to nonlinear dynamical systems identification. Math Comput Simul 79(7):2051–2063MATHMathSciNetCrossRef Lu Z, Sun J, Butts KR (2009) Linear programming support vector regression with wavelet kernel: a new approach to nonlinear dynamical systems identification. Math Comput Simul 79(7):2051–2063MATHMathSciNetCrossRef
Zurück zum Zitat Mangasarian OL, Wild EW (2007) Nonlinear knowledge in kernel approximation. IEEE Trans Neural Netw 18(1):300–306CrossRef Mangasarian OL, Wild EW (2007) Nonlinear knowledge in kernel approximation. IEEE Trans Neural Netw 18(1):300–306CrossRef
Zurück zum Zitat Mingqing Hu, Chen Yiqiang (2009) Building sparse multiple-kernel SVM classifiers. IEEE Trans Neural Netw 20(5):827–839CrossRef Mingqing Hu, Chen Yiqiang (2009) Building sparse multiple-kernel SVM classifiers. IEEE Trans Neural Netw 20(5):827–839CrossRef
Zurück zum Zitat Muller KR, Mika S, Ratsch G, Tsuda K, Scholkopf B (2001) An introduction to kernel-based learning algorithms. IEEE Trans Neural Netw 12(2):181–201CrossRef Muller KR, Mika S, Ratsch G, Tsuda K, Scholkopf B (2001) An introduction to kernel-based learning algorithms. IEEE Trans Neural Netw 12(2):181–201CrossRef
Zurück zum Zitat Nguyen C-V, Tay DBH (2008) Regression using multikernel and semiparametric support vector algorithms. IEEE Signal Process Lett 15:481–484CrossRef Nguyen C-V, Tay DBH (2008) Regression using multikernel and semiparametric support vector algorithms. IEEE Signal Process Lett 15:481–484CrossRef
Zurück zum Zitat Mangasarian Olvi L, Shavlik Jude W (2004) Knowledge-based kernel approximation. J Mach Learn Res 5:1127–1141MATH Mangasarian Olvi L, Shavlik Jude W (2004) Knowledge-based kernel approximation. J Mach Learn Res 5:1127–1141MATH
Zurück zum Zitat Pasolli L, Notarnicola C, Bruzzone L (2012) Multi-objective parameter optimization in support vector regression: general formulation and application to the retrieval of soil moisture from remote sensing data. IEEE J Select Topics Appl Earth Observ Remote Sens 5(5):1495–1508CrossRef Pasolli L, Notarnicola C, Bruzzone L (2012) Multi-objective parameter optimization in support vector regression: general formulation and application to the retrieval of soil moisture from remote sensing data. IEEE J Select Topics Appl Earth Observ Remote Sens 5(5):1495–1508CrossRef
Zurück zum Zitat Phienthrakul T, Kijsirikul B (2010) Evolutionary strategies for hyperparameters of support vector machines based on multi-scale radial basis function kernels. Soft Comput 14(7):681–699CrossRef Phienthrakul T, Kijsirikul B (2010) Evolutionary strategies for hyperparameters of support vector machines based on multi-scale radial basis function kernels. Soft Comput 14(7):681–699CrossRef
Zurück zum Zitat Pozdnoukhov A, Kanevski M (2008) Multi-scale support vector algorithms for hot spot detection and modelling. Stoch Environ Res Risk Assess 22(5):647–660MathSciNetCrossRef Pozdnoukhov A, Kanevski M (2008) Multi-scale support vector algorithms for hot spot detection and modelling. Stoch Environ Res Risk Assess 22(5):647–660MathSciNetCrossRef
Zurück zum Zitat Qiu S, Lane T (2009) A framework for multiple kernel support vector regression and its applications to siRNA efficacy prediction. IEEE/ACM Trans Comput Biol Bioinform 6(2):190–199CrossRef Qiu S, Lane T (2009) A framework for multiple kernel support vector regression and its applications to siRNA efficacy prediction. IEEE/ACM Trans Comput Biol Bioinform 6(2):190–199CrossRef
Zurück zum Zitat Rakotomamonjy A, Bach FR, Canu S, Grandvalet Y (2008) SimpleMKL. J Mach Learn Res 9:2491–2521MATHMathSciNet Rakotomamonjy A, Bach FR, Canu S, Grandvalet Y (2008) SimpleMKL. J Mach Learn Res 9:2491–2521MATHMathSciNet
Zurück zum Zitat Sagiroglu S, Guney K, Erler M (1999) Calculation of bandwidth for electrically thin and thick rectangular microstrip antennas with the use of multilayered perceptrons. Int J RF Microwave Comput Aided Eng 9(3):277–286CrossRef Sagiroglu S, Guney K, Erler M (1999) Calculation of bandwidth for electrically thin and thick rectangular microstrip antennas with the use of multilayered perceptrons. Int J RF Microwave Comput Aided Eng 9(3):277–286CrossRef
Zurück zum Zitat Sanchez AVD (2003) Advanced support vector machines and kernel methods. Neurocomputing 55(1–2):5–20CrossRef Sanchez AVD (2003) Advanced support vector machines and kernel methods. Neurocomputing 55(1–2):5–20CrossRef
Zurück zum Zitat Smola AJ, Sch\(\ddot{\rm o}\)lkopf B (2004) A tutorial on support vector regression. Stat Comput 14:199–220 Smola AJ, Sch\(\ddot{\rm o}\)lkopf B (2004) A tutorial on support vector regression. Stat Comput 14:199–220
Zurück zum Zitat Smola A, Schoelkopf B, Raetsch G (1999) Linear programs for automatic accuracy control in regression. In: Proceedings of the Ninth International Conference on Artificial Neural Networks, Edinburgh, UK. IEE Conference Publication. IEE, pp 575–580 Smola A, Schoelkopf B, Raetsch G (1999) Linear programs for automatic accuracy control in regression. In: Proceedings of the Ninth International Conference on Artificial Neural Networks, Edinburgh, UK. IEE Conference Publication. IEE, pp 575–580
Zurück zum Zitat Smola A, Scholkopf B (2002) Learning with Kernels. MIT Press, Cambridge Smola A, Scholkopf B (2002) Learning with Kernels. MIT Press, Cambridge
Zurück zum Zitat Sonnenburg S, Ratsch G, Schafer C, Scholkopf B (2006) Large scale multiple kernel learning. J Mach Learn Res 7:1531–1565MATHMathSciNet Sonnenburg S, Ratsch G, Schafer C, Scholkopf B (2006) Large scale multiple kernel learning. J Mach Learn Res 7:1531–1565MATHMathSciNet
Zurück zum Zitat Subrahmanya N, Shin YC (2010) Sparse multiple kernel learning for signal processing applications. IEEE Trans Pattern Anal Mach Intell 32(5):788–798 Subrahmanya N, Shin YC (2010) Sparse multiple kernel learning for signal processing applications. IEEE Trans Pattern Anal Mach Intell 32(5):788–798
Zurück zum Zitat Varma M, Babu BR (2009) More generality in efficient multiple kernel learning. In, Montreal, QC, Canada, 2009. Proceedings of the 26th International Conference on machine learning, ICML. Omnipress, pp 1065–1072 Varma M, Babu BR (2009) More generality in efficient multiple kernel learning. In, Montreal, QC, Canada, 2009. Proceedings of the 26th International Conference on machine learning, ICML. Omnipress, pp 1065–1072
Zurück zum Zitat Zhao YP, Sun JG (2011) Multikernel semiparametric linear programming support vector regression. Expert Systems Appl 38:1611–1618 Zhao YP, Sun JG (2011) Multikernel semiparametric linear programming support vector regression. Expert Systems Appl 38:1611–1618
Zurück zum Zitat Yu Y, Qian F (2008) Multi-scale linear programming support vector regression for ethylene distillation modeling. In: Chongqing, China. Proceedings of the World Congress on Intelligent Control and Automation (WCICA). Institute of Electrical and Electronics Engineers Inc., pp 1548–1552 Yu Y, Qian F (2008) Multi-scale linear programming support vector regression for ethylene distillation modeling. In: Chongqing, China. Proceedings of the World Congress on Intelligent Control and Automation (WCICA). Institute of Electrical and Electronics Engineers Inc., pp 1548–1552
Zurück zum Zitat Zheng D, Wang J, Zhao Y (2006) Non-flat function estimation with a multi-scale support vector regression. Neurocomputing 70(1–3): 420–429 Zheng D, Wang J, Zhao Y (2006) Non-flat function estimation with a multi-scale support vector regression. Neurocomputing 70(1–3): 420–429
Zurück zum Zitat Zhou J, Huang J, Xue X (2011) Modeling and optimization for electromechanical coupling of cavity filters based on support vector regression. (Dianzi Yu Xinxi Xuebao) J Electron Inf Technol 33(11):2780–2784 Zhou J, Huang J, Xue X (2011) Modeling and optimization for electromechanical coupling of cavity filters based on support vector regression. (Dianzi Yu Xinxi Xuebao) J Electron Inf Technol 33(11):2780–2784
Zurück zum Zitat Zhou J, Duan B, Huang J (2010) Influence and tuning of tunable screws for microwave filters using least squares support vector regression. Int J RF Microwave Comput Aided Eng 20:422–429 Zhou J, Duan B, Huang J (2010) Influence and tuning of tunable screws for microwave filters using least squares support vector regression. Int J RF Microwave Comput Aided Eng 20:422–429
Zurück zum Zitat Zhou J, Huang J (2010) Incorporating priori knowledge into linear programming support vector regression. In: 2010 IEEE International Conference on Intelligent Computing and Integrated Systems, ICISS2010, October 22, 2010–October 24, 2010, Guilin, China. IEEE Computer Society, pp 591–595 Zhou J, Huang J (2010) Incorporating priori knowledge into linear programming support vector regression. In: 2010 IEEE International Conference on Intelligent Computing and Integrated Systems, ICISS2010, October 22, 2010–October 24, 2010, Guilin, China. IEEE Computer Society, pp 591–595
Zurück zum Zitat Zhou J, Huang J (2013) Intelligent tuning for microwave filters based on multi-kernel machine learning model. In: 5th IEEE International Symposium on Microwave, Antenna, Propagation and EMC Technologies for Wireless Communications, MAPE 2013, October 29, 2013–October 31, 2013. Chengdu, China, pp 259–266 Zhou J, Huang J (2013) Intelligent tuning for microwave filters based on multi-kernel machine learning model. In: 5th IEEE International Symposium on Microwave, Antenna, Propagation and EMC Technologies for Wireless Communications, MAPE 2013, October 29, 2013–October 31, 2013. Chengdu, China, pp 259–266
Metadaten
Titel
Incorporating prior knowledge and multi-kernel into linear programming support vector regression
verfasst von
Jinzhu Zhou
Baoyan Duan
Jin Huang
Na Li
Publikationsdatum
01.07.2015
Verlag
Springer Berlin Heidelberg
Erschienen in
Soft Computing / Ausgabe 7/2015
Print ISSN: 1432-7643
Elektronische ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-014-1390-x

Weitere Artikel der Ausgabe 7/2015

Soft Computing 7/2015 Zur Ausgabe