Skip to main content

2015 | OriginalPaper | Buchkapitel

21. Bounding Embeddings of VC Classes into Maximum Classes

verfasst von : J. Hyam Rubinstein, Benjamin I. P. Rubinstein, Peter L. Bartlett

Erschienen in: Measures of Complexity

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

One of the earliest conjectures in computational learning theory—the Sample Compression conjecture—asserts that concept classes (equivalently set systems) admit compression schemes of size linear in their VC dimension.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
As discussed below, we consider concept classes evaluated on finite samples. Such projections are equivalent to subsets of the n-cube. Thus we discuss concept classes as such subsets without loss of generality.
 
2
We use \(\mathbf {1}\left[ p\right] \) to denote the indicator function on predicate p, and [n] to denote integers \(\{1,\ldots ,n\}\).
 
Literatur
1.
Zurück zum Zitat Abraham, I., Delling, D., Fiat, A., Goldberg, A.V., Werneck, R.F.: VC-dimension and shortest path algorithms. In: ICALP’11. Lecture Notes in Computer Science, vol. 6755, pp. 690–699. Springer, Berlin (2011) Abraham, I., Delling, D., Fiat, A., Goldberg, A.V., Werneck, R.F.: VC-dimension and shortest path algorithms. In: ICALP’11. Lecture Notes in Computer Science, vol. 6755, pp. 690–699. Springer, Berlin (2011)
2.
Zurück zum Zitat Angluin, D.: Computational learning theory: survey and selected bibliography. In: STOC’92, pp. 351–369. ACM (1992) Angluin, D.: Computational learning theory: survey and selected bibliography. In: STOC’92, pp. 351–369. ACM (1992)
3.
Zurück zum Zitat Anthony, M., Bartlett, P.L.: Neural Network Learning: Theoretical Foundations. Cambridge University Press, Cambridge (1999) Anthony, M., Bartlett, P.L.: Neural Network Learning: Theoretical Foundations. Cambridge University Press, Cambridge (1999)
4.
Zurück zum Zitat Ben-David, S., Litman, A.: Combinatorial variability of Vapnik-Chervonenkis classes with applications to sample compression schemes. Discret. Appl. Math. 86(1), 3–25 (1998)MathSciNetCrossRefMATH Ben-David, S., Litman, A.: Combinatorial variability of Vapnik-Chervonenkis classes with applications to sample compression schemes. Discret. Appl. Math. 86(1), 3–25 (1998)MathSciNetCrossRefMATH
5.
Zurück zum Zitat Blumer, A., Ehrenfeucht, A., Haussler, D., Warmuth, M.: Learnability and the Vapnik-Chervonenkis dimension. J. ACM 36(4), 929–965 (1989)MathSciNetCrossRefMATH Blumer, A., Ehrenfeucht, A., Haussler, D., Warmuth, M.: Learnability and the Vapnik-Chervonenkis dimension. J. ACM 36(4), 929–965 (1989)MathSciNetCrossRefMATH
6.
Zurück zum Zitat Brönnimann, H., Goodrich, M.T.: Almost optimal set covers in finite VC-dimension. Discret. Comput. Geom. 14(1), 463–479 (1995)CrossRefMATH Brönnimann, H., Goodrich, M.T.: Almost optimal set covers in finite VC-dimension. Discret. Comput. Geom. 14(1), 463–479 (1995)CrossRefMATH
7.
Zurück zum Zitat Devroye, L., Györfi, L., Lugosi, G.: A Probabilistic Theory of Pattern Recognition. Springer, Berlin (1996) Devroye, L., Györfi, L., Lugosi, G.: A Probabilistic Theory of Pattern Recognition. Springer, Berlin (1996)
8.
Zurück zum Zitat Doliwa, T., Simon, H.U., Zilles, S.: Recursive teaching dimension, learning complexity, and maximum classes. In: ALT’10. Lecture Notes in Computer Science, vol. 6331, pp. 209–223 (2010) Doliwa, T., Simon, H.U., Zilles, S.: Recursive teaching dimension, learning complexity, and maximum classes. In: ALT’10. Lecture Notes in Computer Science, vol. 6331, pp. 209–223 (2010)
9.
Zurück zum Zitat Ehrenfeucht, A., Haussler, D., Kearns, M.J., Valiant, L.G.: A general lower bound on the number of examples needed for learning. Inf. Comput. 82(3), 247–261 (1989)MathSciNetCrossRefMATH Ehrenfeucht, A., Haussler, D., Kearns, M.J., Valiant, L.G.: A general lower bound on the number of examples needed for learning. Inf. Comput. 82(3), 247–261 (1989)MathSciNetCrossRefMATH
10.
Zurück zum Zitat Floyd, S.: On space-bounded learning and the Vapnik-Chervonenkis dimension. Technical report TR-89-061, ICSI, UC Berkeley (1989) Floyd, S.: On space-bounded learning and the Vapnik-Chervonenkis dimension. Technical report TR-89-061, ICSI, UC Berkeley (1989)
11.
Zurück zum Zitat Guruswami, V., Hastad, J., Kopparty, S.: On the list-decodability of random linear codes. In: STOC’10, pp. 409–416. ACM (2010) Guruswami, V., Hastad, J., Kopparty, S.: On the list-decodability of random linear codes. In: STOC’10, pp. 409–416. ACM (2010)
12.
Zurück zum Zitat Haussler, D.: Probably approximately correct learning. In: AAAI’90, pp. 1101–1108 (1990) Haussler, D.: Probably approximately correct learning. In: AAAI’90, pp. 1101–1108 (1990)
13.
Zurück zum Zitat Haussler, D.: Sphere packing numbers for subsets of the boolean \(n\)-cube with bounded Vapnik-Chervonenkis dimension. J. Comb. Theory Ser. A 69(2), 217–232 (1995)MathSciNetCrossRefMATH Haussler, D.: Sphere packing numbers for subsets of the boolean \(n\)-cube with bounded Vapnik-Chervonenkis dimension. J. Comb. Theory Ser. A 69(2), 217–232 (1995)MathSciNetCrossRefMATH
14.
Zurück zum Zitat Haussler, D., Littlestone, N., Warmuth, M.: Predicting \(\{0,1\}\) functions on randomly drawn points. Inf. Comput. 115(2), 284–293 (1994)MathSciNetCrossRef Haussler, D., Littlestone, N., Warmuth, M.: Predicting \(\{0,1\}\) functions on randomly drawn points. Inf. Comput. 115(2), 284–293 (1994)MathSciNetCrossRef
15.
Zurück zum Zitat Haussler, D., Welzl, E.: Epsilon-nets and simplex range queries. In: SOGC’86, pp. 61–71. ACM (1986) Haussler, D., Welzl, E.: Epsilon-nets and simplex range queries. In: SOGC’86, pp. 61–71. ACM (1986)
16.
Zurück zum Zitat Kleinberg, J.M.: Two algorithms for nearest-neighbor search in high dimensions. In: STOC’97, pp. 599–608. ACM (1997) Kleinberg, J.M.: Two algorithms for nearest-neighbor search in high dimensions. In: STOC’97, pp. 599–608. ACM (1997)
17.
Zurück zum Zitat Kuzmin, D., Warmuth, M.: Unlabeled compression schemes for maximum classes. J. Mach. Learn. Res. 8, 2047–2081 (2007)MathSciNetMATH Kuzmin, D., Warmuth, M.: Unlabeled compression schemes for maximum classes. J. Mach. Learn. Res. 8, 2047–2081 (2007)MathSciNetMATH
19.
Zurück zum Zitat Livni, R., Simon, P.: Honest compressions and their application to compression schemes. In: COLT’13 (2013) Livni, R., Simon, P.: Honest compressions and their application to compression schemes. In: COLT’13 (2013)
20.
Zurück zum Zitat Matoušek, J.: Geometric range searching. ACM Comput. Surv. 26(4), 421–461 (1994) Matoušek, J.: Geometric range searching. ACM Comput. Surv. 26(4), 421–461 (1994)
21.
Zurück zum Zitat Procaccia, A.D., Rosenschein, J.S.: Exact VC dimension of monotone formulas. Neural Inf. Process. Lett. Rev. 10(7), 165–168 (2006) Procaccia, A.D., Rosenschein, J.S.: Exact VC dimension of monotone formulas. Neural Inf. Process. Lett. Rev. 10(7), 165–168 (2006)
22.
Zurück zum Zitat Rubinstein, B.I.P., Bartlett, P.L., Rubinstein, J.H.: Shifting: one-inclusion mistake bounds and sample compression. J. Comput. Syst. Sci.: Spec. Issue Learn. Theory 2006 75(1), 37–59 (2009) Rubinstein, B.I.P., Bartlett, P.L., Rubinstein, J.H.: Shifting: one-inclusion mistake bounds and sample compression. J. Comput. Syst. Sci.: Spec. Issue Learn. Theory 2006 75(1), 37–59 (2009)
23.
Zurück zum Zitat Rubinstein, B.I.P., Rubinstein, J.H.: Geometric & topological representations of maximum classes with applications to sample compression. In: COLT’08, pp. 299–310 (2008) Rubinstein, B.I.P., Rubinstein, J.H.: Geometric & topological representations of maximum classes with applications to sample compression. In: COLT’08, pp. 299–310 (2008)
24.
Zurück zum Zitat Rubinstein, B.I.P., Rubinstein, J.H.: A geometric approach to sample compression. J. Mach. Learn. Res. 13(1), 1221–1261 (2012)MathSciNetMATH Rubinstein, B.I.P., Rubinstein, J.H.: A geometric approach to sample compression. J. Mach. Learn. Res. 13(1), 1221–1261 (2012)MathSciNetMATH
25.
Zurück zum Zitat Sauer, N.: On the density of families of sets. J. Comb. Theory Ser. A 13(1), 145–147 (1972) Sauer, N.: On the density of families of sets. J. Comb. Theory Ser. A 13(1), 145–147 (1972)
26.
Zurück zum Zitat Shelah, S.: A combinatorial problem; stability and order for models and theories in infinitary languages. Pac. J. Math. 41(1), 247–261 (1972)CrossRefMATH Shelah, S.: A combinatorial problem; stability and order for models and theories in infinitary languages. Pac. J. Math. 41(1), 247–261 (1972)CrossRefMATH
27.
Zurück zum Zitat Trudeau, R.J.: Introduction to Graph Theory. Dover, New York (1994) Trudeau, R.J.: Introduction to Graph Theory. Dover, New York (1994)
28.
Zurück zum Zitat van der Vaart, A.W., Wellner, J.A.: Weak Convergence and Empirical Processes. Springer, Berlin (1996) van der Vaart, A.W., Wellner, J.A.: Weak Convergence and Empirical Processes. Springer, Berlin (1996)
29.
Zurück zum Zitat Vapnik, V.N., Chervonenkis, A.Y.: On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab. Appl. 16(2), 264–280 (1971) (This volume, Chap. 3) Vapnik, V.N., Chervonenkis, A.Y.: On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab. Appl. 16(2), 264–280 (1971) (This volume, Chap. 3)
30.
Zurück zum Zitat von Luxburg, U., Bousquet, O., Schölkopf, B.: A compression approach to support vector model selection. J. Mach. Learn. Res. 5, 293–323 (2004) von Luxburg, U., Bousquet, O., Schölkopf, B.: A compression approach to support vector model selection. J. Mach. Learn. Res. 5, 293–323 (2004)
31.
Zurück zum Zitat Warmuth, M.K.: Compressing to VC dimension many points. In: COLT’03, Learning Theory and Kernel Machines. Lecture Notes in Computer Science, vol. 2777, pp. 743–744 (2003) Warmuth, M.K.: Compressing to VC dimension many points. In: COLT’03, Learning Theory and Kernel Machines. Lecture Notes in Computer Science, vol. 2777, pp. 743–744 (2003)
32.
Zurück zum Zitat Welzl, E.: Complete range spaces (1987) (Unpublished notes) Welzl, E.: Complete range spaces (1987) (Unpublished notes)
33.
Zurück zum Zitat Welzl, E., Wöginger, G.: On Vapnik-Chervonenkis dimension one (1987) (Unpublished notes) Welzl, E., Wöginger, G.: On Vapnik-Chervonenkis dimension one (1987) (Unpublished notes)
Metadaten
Titel
Bounding Embeddings of VC Classes into Maximum Classes
verfasst von
J. Hyam Rubinstein
Benjamin I. P. Rubinstein
Peter L. Bartlett
Copyright-Jahr
2015
DOI
https://doi.org/10.1007/978-3-319-21852-6_21