nach oben

Erschienen in:

2017 | OriginalPaper | Buchkapitel

Bayesian Nonlinear Support Vector Machines for Big Data

verfasst von : Florian Wenzel, Théo Galy-Fajou, Matthäus Deutsch, Marius Kloft

Erschienen in: Machine Learning and Knowledge Discovery in Databases

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

We propose a fast inference method for Bayesian nonlinear support vector machines that leverages stochastic variational inference and inducing points. Our experiments show that the proposed method is faster than competing Bayesian approaches and scales easily to millions of data points. It provides additional features over frequentist competitors such as accurate predictive uncertainty estimates and automatic hyperparameter search.

Code related to this chapter is available at: https://doi.org/10.6084/m9.figshare.5443627

Data related to this chapter are available at: https://doi.org/10.6084/m9.figshare.5443624 and https://doi.org/10.6084/m9.figshare.5443621

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel SetExpan: Corpus-Based Set Expansion via Context Feature Selection and Rank Ensemble

Nächstes Kapitel Entropic Trace Estimates for Log Determinants

Note that frequentist approaches can also lead to other forms of uncertainty estimates, e.g. in form of confidence intervals. But since the classic SVM does not exhibit a probabilistic formulation these uncertainty estimates cannot be directly computed.

This follows directly since \(K_{mm}\) and \(A^{-\frac{1}{2}}\) are positive definite.

The RBF kernel is defined as \(k(x_1,x_2,\theta )=\exp \left( -\frac{||x_1-x_2||}{\theta ^2}\right) \), where \(\theta \) is the length scale parameter.

For a comparison with the stochastic variational inference version of GPC, see Sect. 5.3.

The length scale parameter tuning is not included in the training time. We found \(\theta = 5.0\) by our proposed automatic tuning approach.

Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)MATH

Polson, N.G., Scott, S.L.: Data augmentation for support vector machines. Bayesian Anal. 6(1), 1–24 (2011)MathSciNetCrossRefMATH

Henao, R., Yuan, X., Carin, L.: Bayesian nonlinear support vector machines and discriminative factor modeling. In: NIPS (2014)

Fernández-Delgado, M., Cernadas, E., Barro, S., Amorim, D.: Do we need hundreds of classifiers to solve real world classification problems? JMLR 15(1), 3133–3181 (2014)MathSciNetMATH

Mohri, M., Rostamizadeh, A., Talwalkar, A.: Foundations of Machine Learning. MIT press, Cambridge (2012)MATH

Hoffman, M.D., Blei, D.M., Wang, C., Paisley, J.: Stochastic variational inference. JMLR 14, 1303–1347 (2013)MathSciNetMATH

Hensman, J., Fusi, N., Lawrence, N.D.: Gaussian processes for big data. In: Conference on Uncertainty in Artificial Intellegence (2013)

Platt, P.J.C.: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv. Large Margin Classif. 10(3), 61–74 (1999)

Rasmussen, C.E., Williams, C.K.I.: Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). MIT Press, Cambridge (2005)

10.

Hensman, J., Matthews, A.: Scalable variational Gaussian process classification. In: AISTATS (2015)

11.

Baldi, P., Sadowski, P., Whiteson, D.: Searching for exotic particles in high-energy physics with deep learning. Nature Commun. 4 (2014). Article no. 4308

12.

Zhu, J., Chen, N., Perkins, H., Zhang, B.: Gibbs max-margin topic models with data augmentation. JMLR 15(1), 1073–1110 (2014)MathSciNetMATH

13.

Xu, M., Zhu, J., Zhang, B.: Fast max-margin matrix factorization with data augmentation. In: ICML, pp. 978–986 (2013)

14.

Zhang, A., Zhu, J., Zhang, B.: Max-margin infinite hidden Markov models. In: ICML (2014)

15.

Luts, J., Ormerod, J.T.: Mean field variational Bayesian inference for support vector machine classification. Comput. Stat. Data Anal. 73, 163–176 (2014)MathSciNetCrossRef

16.

Snelson, E., Ghahramani, Z.: Sparse GPs using pseudo-inputs. In: NIPS (2006)

17.

Kloft, M., Brefeld, U., Sonnenburg, S., Zien, A.: \(lp\)-norm multiple kernel learning. JMLR 12, 953–997 (2011)MATH

18.

Jordan, M.I., Ghahramani, Z., Jaakkola, T.S., Saul, L.K.: An introduction to variational methods for graphical models. Mach. Learn. 37(2), 183–233 (1999)CrossRefMATH

19.

Wainwright, M.J., Jordan, M.I.: Graphical models, exponential families, and variational inference. Found. Trends Mach. Learn. 1(1–2), 1–305 (2008)MATH

20.

Jørgensen, B.: Statistical Properties of the Generalized Inverse Gaussian Distribution. Springer, New York (2012). https://doi.org/10.1007/978-1-4612-5698-4

21.

Amari, S., Nagaoka, H.: Methods of Information Geometry. American Mathematical Society, Providence (2007)MATH

22.

Martens, J.: New insights and perspectives on the natural gradient method. Arxiv Preprint (2017)

23.

Amari, S.: Natural gradient works efficiently in learning. Neural Comput. 10, 251–276 (1998)CrossRef

24.

Titsias, M.K.: Variational learning of inducing variables in sparse Gaussian processes. In: Artificial Intelligence and Statistics, vol. 12, pp. 567–574 (2009)

25.

Murphy, K.P.: Machine Learning: A Probabilistic Perspective. The MIT Press, Cambridge (2012)MATH

26.

Ranganath, R., Wang, C., Blei, D.M., Xing, E.P.: An adaptive learning rate for stochastic variational inference. In: ICML (2013)

27.

Maritz, J., Lwin, T.: Empirical Bayes Methods with Applications: Monographs on Statistics and Applied Probability. Chapman & Hall/CRC, Boca Raton (1989)MATH

28.

Mandt, S., Hoffman, M., Blei, D.: A variational analysis of stochastic gradient algorithms. In: ICML (2016)

29.

Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 27:1–27:27 (2011)CrossRef

30.

Brier, G.W.: Verification of forecasts expressed in terms of probability. Mon. Weather Rev. 78(1), 1–3 (1950)CrossRef

31.

Diethe, T.: 13 benchmark datasets derived from the UCI, DELVE and STATLOG repositories (2015)

32.

Bachem, O., Lucic, M., Hassani, H., Krause, A.: Fast and provably good seedings for k-means. In: NIPS (2016)

33.

Lichman, M.: UCI machine learning repository (2013)

34.

Mandt, S., Wenzel, F., Nakajima, S., Cunningham, J.P., Lippert, C., Kloft, M.: Sparse probit linear mixed model. Mach. Learn. 106(9–10), 1621–1642 (2017)MathSciNetCrossRef

35.

Perdisci, R., Gu, G., Lee, W.: Using an ensemble of one-class SVM classifiers to H. P.-based anomaly detection systems. In: Data Mining (2006)

Titel: Bayesian Nonlinear Support Vector Machines for Big Data
verfasst von: Florian Wenzel
Théo Galy-Fajou
Matthäus Deutsch
Marius Kloft
Verlag: Springer International Publishing
Buch: Machine Learning and Knowledge Discovery in Databases
Print ISBN: 978-3-319-71248-2

Electronic ISBN: 978-3-319-71249-9

Copyright-Jahr: 2017
DOI: https://doi.org/10.1007/978-3-319-71249-9_19

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"