Skip to main content

2017 | OriginalPaper | Buchkapitel

Hyper-Parameter Tuning for Support Vector Machines by Estimation of Distribution Algorithms

verfasst von : Luis Carlos Padierna, Martín Carpio, Alfonso Rojas, Héctor Puga, Rosario Baltazar, Héctor Fraire

Erschienen in: Nature-Inspired Design of Hybrid Intelligent Systems

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Hyper-parameter tuning for support vector machines has been widely studied in the past decade. A variety of metaheuristics, such as Genetic Algorithms and Particle Swarm Optimization have been considered to accomplish this task. Notably, exhaustive strategies such as Grid Search or Random Search continue to be implemented for hyper-parameter tuning and have recently shown results comparable to sophisticated metaheuristics. The main reason for the success of exhaustive techniques is due to the fact that only two or three parameters need to be adjusted when working with support vector machines. In this chapter, we analyze two Estimation Distribution Algorithms, the Univariate Marginal Distribution Algorithm and the Boltzmann Univariate Marginal Distribution Algorithm, to verify if these algorithms preserve the effectiveness of Random Search and at the same time make more efficient the process of finding the optimal hyper-parameters without increasing the complexity of Random Search.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat N. Christianini and J. Shawe-Taylor. An Introduction to SVM and other Kernel Based Methods. Cambridge, U.K.: Cambridge University Press, 2000. N. Christianini and J. Shawe-Taylor. An Introduction to SVM and other Kernel Based Methods. Cambridge, U.K.: Cambridge University Press, 2000.
2.
Zurück zum Zitat V. Vapnik, Statistical Learning Theory, New York: John Wiley and Sons, 1998. V. Vapnik, Statistical Learning Theory, New York: John Wiley and Sons, 1998.
3.
Zurück zum Zitat M. Kanevski, V. Timonin and A. Pozdnukhov, Machine Learning for Spatial Environmental Data: theory, applications and software, CRC Press, 2009. M. Kanevski, V. Timonin and A. Pozdnukhov, Machine Learning for Spatial Environmental Data: theory, applications and software, CRC Press, 2009.
4.
Zurück zum Zitat O. Chapelle, V. Vapnik, O. Bousquet and S. Mukherjee, Choosing Multiple Parameters for Support Vector Machines Machine Learning, vol. 46 (1-3), pp. 131-159, 2002. O. Chapelle, V. Vapnik, O. Bousquet and S. Mukherjee, Choosing Multiple Parameters for Support Vector Machines Machine Learning, vol. 46 (1-3), pp. 131-159, 2002.
5.
Zurück zum Zitat F. Friedrichs and C. Igel. Evolutionary tuning of multiple SVM parameters. Neurocomputing, vol 64 pp. 107-117, 2005. F. Friedrichs and C. Igel. Evolutionary tuning of multiple SVM parameters. Neurocomputing, vol 64 pp. 107-117, 2005.
6.
Zurück zum Zitat C. Gold, A. Holub y P. Sollich. Bayesian approach to feature selection and parameter tuning for support vector machine classifiers. Neural Networks, vol 18 (5) pp. 693-701, 2005. C. Gold, A. Holub y P. Sollich. Bayesian approach to feature selection and parameter tuning for support vector machine classifiers. Neural Networks, vol 18 (5) pp. 693-701, 2005.
7.
Zurück zum Zitat C.-L. Huang and W. Chieh-Jen. A GA-based feature selection and parameters optimization for support vector machines. Expert System with Applications, vol 31(2) pp. 231-240, 2006. C.-L. Huang and W. Chieh-Jen. A GA-based feature selection and parameters optimization for support vector machines. Expert System with Applications, vol 31(2) pp. 231-240, 2006.
8.
Zurück zum Zitat S.-W. Lin, K.-C. Ying y S.-C. L. Z.-J. Chen. Particle Swarm Optimization for Parameter Determination and Feature Selection of Support Vector Machines. Expert System with Applications, vol 35(4) pp. 1817-1824, 2008. S.-W. Lin, K.-C. Ying y S.-C. L. Z.-J. Chen. Particle Swarm Optimization for Parameter Determination and Feature Selection of Support Vector Machines. Expert System with Applications, vol 35(4) pp. 1817-1824, 2008.
9.
Zurück zum Zitat J. Bergstra and Y. Bengio. Random Search for Hyper-Parameter Optimization. Journal of Machine Learning Research, 13(1) pp. 281-305, 2012. J. Bergstra and Y. Bengio. Random Search for Hyper-Parameter Optimization. Journal of Machine Learning Research, 13(1) pp. 281-305, 2012.
10.
Zurück zum Zitat R. Mantovani, A. Rossi, J. Vanschoren and B. d.-C. A. Bischl. Effectiveness of Random Search in SVM hyper-parameter tuning. The International Joint Conference on Neural Networks (IJCNN), 2015. R. Mantovani, A. Rossi, J. Vanschoren and B. d.-C. A. Bischl. Effectiveness of Random Search in SVM hyper-parameter tuning. The International Joint Conference on Neural Networks (IJCNN), 2015.
11.
Zurück zum Zitat A. Shigeo, Support Vector Machines for Pattern Classification, New York: Springer, 2010. A. Shigeo, Support Vector Machines for Pattern Classification, New York: Springer, 2010.
12.
Zurück zum Zitat N. Deng, Y. Tian and C. Zhang, Support Vector Machines, Boca Raton: CRC Press, 2013. N. Deng, Y. Tian and C. Zhang, Support Vector Machines, Boca Raton: CRC Press, 2013.
13.
Zurück zum Zitat M. Hauschild and M. Pelikan. An introduction and survey of estimation of distribution algorithms. Swarm and Evolutionary Computation, vol 1(3) pp. 111-128, 2011. M. Hauschild and M. Pelikan. An introduction and survey of estimation of distribution algorithms. Swarm and Evolutionary Computation, vol 1(3) pp. 111-128, 2011.
14.
Zurück zum Zitat D. Simon, Evolutionary Optimization Algorithms: Biologically Inspired and Population-Based Approaches to Computer Intelligence, Hoboken: John Wiley and Sons, 2013. D. Simon, Evolutionary Optimization Algorithms: Biologically Inspired and Population-Based Approaches to Computer Intelligence, Hoboken: John Wiley and Sons, 2013.
15.
Zurück zum Zitat S. I. Valdez, A. Hernández and S. Botello. A Boltzmann based estimation of distribution algorithm. Information Sciences, vol 236 pp. 126-137, 2013. S. I. Valdez, A. Hernández and S. Botello. A Boltzmann based estimation of distribution algorithm. Information Sciences, vol 236 pp. 126-137, 2013.
Metadaten
Titel
Hyper-Parameter Tuning for Support Vector Machines by Estimation of Distribution Algorithms
verfasst von
Luis Carlos Padierna
Martín Carpio
Alfonso Rojas
Héctor Puga
Rosario Baltazar
Héctor Fraire
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-47054-2_53