Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Chervonenkis’s Recollections Read chapter Read first chapter

2015 | OriginalPaper | Chapter

20. PAC-Bayes Bounds for Supervised Classification

Author : Olivier Catoni

Published in: Measures of Complexity

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

We present in this contribution a synthesis of Seeger’s (PAC-Bayesian generalization error bounds for Gaussian process classification, 2002) and our own (Catoni, PAC-Bayesian Supervised Classification: The Thermodynamics of Statistical Learning, 2007) approach of PAC-Bayes inequalities for 0–1 loss functions. We apply it to supervised classification, and more specifically to the proof of new margin bounds for support vector machines, in the spirit of the bounds established by Langford and Shawe-Taylor (Advances in Neural Information Processing Systems, 2002) and McAllester (Learning Theory and Kernel Machines, COLT 2003).

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Interpretation of Black-Box Predictive Models
next chapter Bounding Embeddings of VC Classes into Maximum Classes
Footnotes
1
We will assume that \(\rho \) is a regular conditional probability kernel, meaning that for any measurable set A the map \((w_1, \dots , w_n) \mapsto \rho (w_1, \dots , w_n)(A)\) is assumed to be measurable. We will also assume that the \(\sigma \)-algebra we consider on \(\varTheta \) is generated by a countable family of subsets. See [1] (p. 50) for more details.
 
Literature
1.
Catoni, O.: Statistical Learning Theory and Stochastic Optimization, Lectures on Probability Theory and Statistics, École d’Été de Probabilités de Saint-Flour XXXI—2001. Lecture Notes in Mathematics, vol. 1851. Springer, Berlin (2004)
2.
Catoni, O.: PAC-Bayesian Supervised Classification: The Thermodynamics of Statistical Learning. IMS Lecture Notes Monograph Series, vol. 56. Institute of Mathematical Statistics, Beachwood (2007)
3.
Germain, P., Lacasse, A., Laviolette, F., Marchand, M.: PAC-Bayesian learning of linear classifiers. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML’09, pp. 353–360. ACM, New York (2009)
4.
Langford, J., Shawe-Taylor, J.: PAC-Bayes & margins. Advances in Neural Information Processing Systems, pp. 423–430. MIT Press, Cambridge (2002)
5.
McAllester, D.: Simplified PAC-Bayesian margin bounds. In: Schölkopf, B., Warmuth, M.K. (eds.) Learning Theory and Kernel Machines, COLT 2003. Lecture Notes in Artificial Intelligence, vol. 2777, pp. 203–215. Springer, Berlin (2003)
6.
McAllester, D.A.: PAC-Bayesian model averaging. In: Proceedings of the 12th Annual Conference on Computational Learning Theory, pp. 164–170. ACM, New York (1999)
7.
McAllester, D.A.: PAC-Bayesian stochastic model selection. Mach. Learn. 51(1), 5–21 (2003)MATHCrossRef
8.
9.
Vapnik, V.: Estimation of Dependences Based on Empirical Data. Springer, Berlin (1982)MATH
10.
Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)MATH
11.
Vapnik, V., Chervonenkis, A.: On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab. Appl. 16(2), 264–280 (1971) (This volume, Chap. 3)
Metadata
Title
PAC-Bayes Bounds for Supervised Classification
Author
Olivier Catoni
Copyright Year
2015
Book
Measures of Complexity

Print ISBN: 978-3-319-21851-9

Electronic ISBN: 978-3-319-21852-6

Copyright Year: 2015

DOI
https://doi.org/10.1007/978-3-319-21852-6_20

Premium Partner

    Image Credits