Abstract
In real-world applications, selecting the appropriate hyper-parameters for support vector machines (SVM) is a difficult and vital step which impacts the generalization capability and classification performance of classifier. In this paper, we analyze the distributing characteristic of hyper-parameters that the optimal hyper-parameters points form neighborhoods. For finding all the optimal points (on the grid points) in neighborhoods, based on this characteristic, we propose a hybrid method that combines evolution strategies (ES) with a grid search (GS), to carry out optimizing selection of these hyperparameters. We firstly use evolution strategies find the optimal points of hyperparameters and secondly execute a grid search in the neighborhood of these points. Our hybrid method takes advantage of the high computing efficiency of ES and the exhaustive searching merit of GS. Experiments show our hybrid method can successfully find the optimal hyper-parameters points in neighborhoods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Cortes, C., Vapnik, V.: Support Vector Network. Machine learning, 20 (1995) 273–297
Friedrichs, F., Igel, C.: Evolutionary Tuning of Multiple SVM Parameters. Neurocmputing, 64 (2005)107–177
Hsu, C., Chang, C., Lin, C.: A Practical Guide to Support Vector Classification, Department of Computer Science and Information Engineering, National Taiwai University (2003)
Chapelle, O., Vapnik, V., Bousquet, O., Mukherjee, S.: Choosing Multiple Parameters for Support Vector Machines. Machine Linearing, 46 (2002) 131–159
Chung, K.-M., Kao, W.-C., Sun, C.-L., Lin, C.-J.: Radius Margin Bounds for Support Vector Machines with RBF Kernel. Neurocmputing, 15 (2003) 2643–2681
Gold, C., Sollich, P.: Model Selection for Support Vector Machine Classification, Neurocomputing, 55 (2003) 221–249
Keerthi, S.-S.: Efficient Tuning of SVM Hyper-Parameters Using Radius/Margin Bound and Iterative Algorithms. IEEE Transactions on Neural Networks, 13 (2002) 1225–1229
Liu, S., Jia, C.-Y. Ma, H.: A New Weighted Support Vector Machine with GA-based Parameter Selection. Proc. the 4th Int’l Conf. on Machine Learning and Cybernetics(ICMLC’05), (Aug. 2005) 18–21
Sergio, A., Rojas, Delmiro, F.-R.: Adapting Multiple Kernel Parameters for Support Vector Machines Using Genetic Algorithms. Division of parasitology national institute for medical research London NW71AA, UK and department of computer science university college London
Hansen, N., Ostermeier, A.: Convergence Properties of Evolution Strategies with the Derandomized Covariance Matrix Adaptation: the (μ/μ,λ)-CMA-ES. Proc. of the 5th European Congress on Intelligent Techniques and Soft Computing(EUFIT’97), (Sept. 1997), 650–654
Hansen, N., Ostermeier, A.: Completely Deranomized Self-adaptation in Evolution Strategies. Evolutionary Computation, 9 (2001) 159–195
Cassabaum, M.-L., Waagen, D.-E., Rodriguez, J.-J., Schmitt, H.-A.: Unsupervised Optimization of Support Vector Machine Parameters, Automatic target recognition XIV, edited by Firooz A. Sadjadi, Proceedings of SPIE, 5426, 316–325
Imbault, F., Lebart, K.: A Stochastic Optimization Approach for Parameter Tuning of Support Vector Machine. Proc. the 17th Int’l conf. on Pattern Recognition (ICPR’04)
Nello, C., John S.-T.: An Introduction to Support Vector Machines and Other Kernelbased Learning Methods. Cambridge university Press. (2000)
Steve, R.-G.: Support Vector Machines for Classification and Regression, Faculty of engineering, science and mathematics school of electronics and computer science. (1998)
Blake, C.-L., Merz, C.-J.: UCI Repository of Machine Learning Databases. http://www.ics.uci.edu/~learn/MLRepository.html. (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Liu, R., Liu, E., Yang, J., Li, M., Wang, F. (2006). Optimizing the Hyper-parameters for SVM by Combining Evolution Strategies with a Grid Search. In: Huang, DS., Li, K., Irwin, G.W. (eds) Intelligent Control and Automation. Lecture Notes in Control and Information Sciences, vol 344. Springer, Berlin, Heidelberg . https://doi.org/10.1007/978-3-540-37256-1_87
Download citation
DOI: https://doi.org/10.1007/978-3-540-37256-1_87
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-37255-4
Online ISBN: 978-3-540-37256-1
eBook Packages: EngineeringEngineering (R0)