Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
An Importance Sampling Approach to the Estimation of Algorithm Performance in Automated Algorithm Design Read chapter Read first chapter

2017 | OriginalPaper | Chapter

Automatic Configuration of Kernel-Based Clustering: An Optimization Approach

Authors : Antonio Candelieri, Ilaria Giordani, Francesco Archetti

Published in: Learning and Intelligent Optimization

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

This paper generalizes a method originally developed by the authors to perform data driven localization of leakages in urban Water Distribution Networks. The method is based on clustering to perform exploratory analysis and a pool of Support Vector Machines to process on line sensors readings. The performance depends on certain hyperparameters which have been considered as decision variables in a sequential model based optimization process. The objective function is related to clustering performance, computed through an external validity index defined according to the leakage localization goal. Thus, as usual in hyperparameters tuning of machine learning algorithms, the objective function is black box. In this paper it is shown how a Bayesian framework offers not only a good performance but also the flexibility to consider in the optimization loop also the automatic configuration of the algorithm. Both Gaussian Processes and Random Forests have been considered to fit the surrogate model of the objective function, while results from a simple grid search have been considered as baseline.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Test Problems for Parallel Algorithms of Constrained Global Optimization
next chapter Solution of the Convergecast Scheduling Problem on a Square Unit Grid When the Transmission Range is 2
Literature
1.
Xia, L., Xiao-dong, W., Xin-hua, Z., Guo-jin, L.: Bayesian theorem based on-line leakage detection and localization of municipal water supply network. Water Wastewater Eng. 12 (2006)
2.
Sivapragasam, C., Maheswaran, R., Venkatesh, V.: ANN-based model for aiding leak detection in water distribution networks. Asian J. Water Environ. Pollut. 5(3), 111–114 (2007)
3.
Xia, L., Guo-jin, L.: Leak detection of municipal water supply network based on the cluster-analysis and fuzzy pattern recognition. In: 2010 International Conference on E-Product E-Service and E-Entertainment (ICEEE), vol. 1(5), pp. 7–9 (2010)
4.
Lijuan, W., Hongwei, Z., Hui, J.: A leak detection method based on EPANET and genetic algorithm in water distribution systems. In: Wu, Y. (ed.) Software Engineering and Knowledge Engineering: Theory and Practice. AISC, vol. 114, pp. 459–465. Springer, Heidelberg (2012). doi:10.​1007/​978-3-642-03718-4_​57 CrossRef
5.
Nasir, A., Soong, B.H., Ramachandran, S.: Framework of WSN based human centric cyber physical in-pipe water monitoring system. In: 11th International Conference on Control, Automation, Robotics and Vision, pp. 1257–1261 (2010)
6.
Soldevila, A., Fernandez-Canti, R.M., Blesa, J., Tornil-Sin, S., Puig, V.: Leak localization in water distribution networks using Bayesian classifiers. J. Process Control 55, 1–9 (2017)CrossRef
7.
Franzin, A., Cáceres, L.P., Stützle, T.: Effect of Transformations of Numerical Parameters in Automatic Algorithm Configuration, IRIDIA Technical Report 2017-006 (2017)
8.
Bagnall, A., Cawley, G.C.: On the Use of Default Parameter Settings in the Empirical Evaluation of Classification Algorithms. arXiv:​1703.​06777v1 [cs.LG] (2017)
9.
Strongin, R.G., Sergeyev, Y.D.: Global Optimization with Non-Convex Constraints - Sequential and Parallel Algorithms. Springer, US (2000)CrossRefMATH
10.
Zhigljavsky, A., Žilinskas, A.: Stochastic Global Optimization. Springer, US (2008)MATH
11.
Locatelli, M., Schoen, F.: Global Optimization - Theory, Algorithms and Applications. MOS-SIAM Series on Optimization. Society for Industrial & Applied Mathematics, Philadelphia (2013)CrossRefMATH
12.
Sergeyev, Y.D., Kvasov, D.E.: Global search based on efficient diagonal partitions and a set of Lipschitz constants. SIAM J. Optim. 16(3), 910–937 (2006)CrossRefMATHMathSciNet
13.
Sergeyev, Y.D., Kvasov, D.E.: A deterministic global optimization using smooth diagonal auxiliary functions. Commun. Nonlinear Sci. Numer. Simul. 21(1), 99–111 (2015)CrossRefMATHMathSciNet
14.
Barkalov, K., Polovinkin, A., Meyerov, I., Sidorov, S., Zolotykh, N.: SVM regression parameters optimization using parallel global search algorithm. In: Malyshkin, V. (ed.) PaCT 2013. LNCS, vol. 7979, pp. 154–166. Springer, Heidelberg (2013). doi:10.​1007/​978-3-642-39958-9_​14 CrossRef
15.
Gillard, J.W., Kvasov, D.E.: Lipschitz optimization methods for fitting a sum of damped sinusoids to a series of observations. Stat. Interface 10, 59–70 (2017)CrossRefMathSciNet
16.
Zabinsky, Z.B.: Stochastic Adaptive Search for Global Optimization, vol. 72. Springer, New York (2013)MATH
17.
Steponavičė, I., Shirazi-Manesh, M., Hyndman, R.J., Smith-Miles, K., Villanova, L.: On sampling methods for costly multiobjective black-box optimization. In: Pardalos, P.M., Zhigljavsky, A., Zilinskas, J. (eds.) Advances in Stochastic and Deterministic Global Optimization. SOIA, vol. 107, pp. 273–296. Springer, Cham (2016). doi:10.​1007/​978-3-319-29975-4_​15 CrossRef
18.
19.
Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., Talwalkar, A.: Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization. arXiv preprint arXiv:​1603.​06560 (2016)
20.
Feurer, M., Klein, A., Eggensperger, K., Springenberg, J., Blum, M., Hutter, F.: Efficient and robust automated machine learning. In: Advances in Neural Information Processing Systems, pp. 2962–2970 (2015)
21.
Shahriari, B., Swersky, K., Wang, Z., Adams, R.P., de Freitas, N.: Taking the human out of the loop: A review of bayesian optimization. Proc. IEEE 104(1), 148–175 (2016)CrossRef
22.
Snoek, J., Larochelle, H., Adams, R.P.: Practical bayesian optimization of machine learning algorithms. In: Advances in Neural Information Processing Systems, pp. 2951–2959 (2012)
23.
Brochu, E., Cora, V.M., De Freitas, N.: A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:​1012.​2599 (2010)
24.
Mockus, J.: On Bayesian methods of optimization. In: Dixon, L.C.W., Szegö, G.P. (eds.) Towards Global Optimization. North-Holland, Amsterdam (1975)
25.
Stützle, T.: Automated algorithm configuration: advances and prospects. In: Camacho, D., Braubach, L., Venticinque, S., Badica, C. (eds.) Intelligent Distributed Computing VIII. SCI, vol. 570, p. 5. Springer, Cham (2015). doi:10.​1007/​978-3-319-10422-5_​2
26.
Mala-Jetmarova, H., Sultanova, N., Savic, D.: Lost in optimization of water distribution systems? a literature review of system operations. Environ. Modell. Softw. 93, 209–254 (2017)CrossRef
27.
Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Coello, C.A.C. (ed.) LION 2011. LNCS, vol. 6683, pp. 507–523. Springer, Heidelberg (2011). doi:10.​1007/​978-3-642-25566-3_​40 CrossRef
28.
Candelieri, A., Soldi, D., Archetti, F.: Cost-effective sensors placement and leak localization - The Neptun pilot of the ICeWater project. J. Water Supply: Res. Technol. AQUA 64(5), 567–582 (2015)CrossRef
29.
Candelieri, A., Soldi, D., Conti, D., Archetti, F.: Analytical leakages localization in water distribution networks through spectral clustering and support vector machines. The icewater approach. Procedia Eng. 89, 1080–1088 (2014)CrossRef
30.
Candelieri, A., Archetti, F., Messina, E.: Improving leakage management in urban water distribution networks through data analytics and hydraulic simulation. WIT Trans. Ecol. Environ. 171, 107–117 (2013)CrossRef
31.
Dhillon, I.S., Guan, Y., Kulis, B.: Kernel k-means: spectral clustering and normalized cuts. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 551–556 (2004)
32.
Žilinskas, A.: On similarities between two models of global optimization: statistical models and radial basis functions. J. Global Optim. 48(1), 173–182 (2010)CrossRefMATHMathSciNet
33.
Arbelaitz, O., Gurrutxaga, I., Muguerza, J., Pérez, J.M., Perona, I.: An extensive comparative study of cluster validity indices. Pattern Recogn. 46(1), 243–256 (2013)CrossRef
34.
Scholkopf, B., Smola, A., Muller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10, 1299–1319 (1998)CrossRef
35.
Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Global Optim. 13(4), 455–492 (1998)CrossRefMATHMathSciNet
36.
Horn, D., Wagner, T., Biermann, D., Weihs, C., Bischl, B.: Model-based multi-objective optimization: taxonomy, multi-point proposal, toolbox and benchmark. In: Gaspar-Cunha, A., Henggeler Antunes, C., Coello, C.C. (eds.) EMO 2015. LNCS, vol. 9018, pp. 64–78. Springer, Cham (2015). doi:10.​1007/​978-3-319-15934-8_​5
37.
Ginsbourger, D., Le Riche, R., Carraro, L.: Kriging is well-suited to parallelize optimization. In: Tenne, Y., Goh, C.K. (eds.) Computational Intelligence in Expensive Optimization Problems. ALO, vol. 2, pp. 131–162. Springer, Heidelberg (2010). doi:10.​1007/​978-3-642-10701-6_​6 CrossRef
38.
Bischl, B., Wessing, S., Bauer, N., Friedrichs, K., Weihs, C.: MOI-MBO: multiobjective infill for parallel model-based optimization. In: Pardalos, Panos M., Resende, M.G.C., Vogiatzis, C., Walteros, J.L. (eds.) LION 2014. LNCS, vol. 8426, pp. 173–186. Springer, Cham (2014). doi:10.​1007/​978-3-319-09584-4_​17
39.
Bergstra, J.S., Bardenet, R., Bengio, Y., Kégl, B.: Algorithms for hyperparameter optimization. In: Advances in Neural Information Processing Systems, pp. 2546–2554 (2011)
40.
Bischl, B., Richter, J., Bossek, J., Horn, D., Thomas, J., Lang, M.: mlrMBO: A Modular Framework for Model-Based Optimization of Expensive Black-Box Functions. arXiv preprint arXiv:​1703.​03373 (2017)
41.
Thornton, C., Hutter, F., Hoos, H.H., Leyton-Brown, K.: Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms. In: Proceedings of ACM SIGKDD, pp. 847–855 (2013)
42.
Lang, M., Kotthaus, H., Marwedel, P., Weihs, C., Rahnenführer, J., Bischl, B.: Automatic model selection for high-dimensional survival analysis. J. Stat. Comput. Simul. 85(1), 62–76 (2015)CrossRefMathSciNet
43.
Horn, D., Bischl, B.: Multi-objective parameter configuration of machine learning algorithms using model-based optimization. In: 2016 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–8 (2016)
44.
Richter, J., Kotthaus, H., Bischl, B., Marwedel, P., Rahnenführer, J., Lang, M.: Faster model-based optimization through resource-aware scheduling strategies. In: Festa, P., Sellmann, M., Vanschoren, J. (eds.) LION 2016. LNCS, vol. 10079, pp. 267–273. Springer, Cham (2016). doi:10.​1007/​978-3-319-50349-3_​22 CrossRef
45.
Kvasov, D.E., Sergeyev, Y.D.: Deterministic approaches for solving practical black-box global optimization problems. Adv. Eng. Softw. 80, 58–66 (2015)CrossRef
46.
Wang, Z., Zoghi, M., Hutter, F., Matheson, D., De Freitas, N.: Bayesian optimization in high dimensions via random embeddings. In: AAAI Press/International Joint Conferences on Artificial Intelligence (2013)
47.
Klein, A., Falkner, S., Bartels, S., Hennig, P., Hutter, F.: Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets. arXiv:​1605.​07079 (2017)
Metadata
Title
Automatic Configuration of Kernel-Based Clustering: An Optimization Approach
Authors
Antonio Candelieri
Ilaria Giordani
Francesco Archetti
Copyright Year
2017
Book
Learning and Intelligent Optimization

Print ISBN: 978-3-319-69403-0

Electronic ISBN: 978-3-319-69404-7

Copyright Year: 2017

DOI
https://doi.org/10.1007/978-3-319-69404-7_3

Premium Partner

    Image Credits