Skip to main content
Erschienen in: Foundations of Computational Mathematics 2/2012

01.04.2012

Learning Functions of Few Arbitrary Linear Parameters in High Dimensions

verfasst von: Massimo Fornasier, Karin Schnass, Jan Vybiral

Erschienen in: Foundations of Computational Mathematics | Ausgabe 2/2012

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Let us assume that f is a continuous function defined on the unit ball of ℝ d , of the form f(x)=g(Ax), where A is a k×d matrix and g is a function of k variables for kd. We are given a budget m∈ℕ of possible point evaluations f(x i ), i=1,…,m, of f, which we are allowed to query in order to construct a uniform approximating function. Under certain smoothness and variation assumptions on the function g, and an arbitrary choice of the matrix A, we present in this paper
1.
a sampling choice of the points {x i } drawn at random for each function approximation;
 
2.
algorithms (Algorithm 1 and Algorithm 2) for computing the approximating function, whose complexity is at most polynomial in the dimension d and in the number m of points.
 
Due to the arbitrariness of A, the sampling points will be chosen according to suitable random distributions, and our results hold with overwhelming probability. Our approach uses tools taken from the compressed sensing framework, recent Chernoff bounds for sums of positive semidefinite matrices, and classical stability bounds for invariant subspaces of singular value decompositions.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat R. Ahlswede, A. Winter, Strong converse for identification via quantum channels, IEEE Trans. Inf. Theory 48(3), 569–579 (2002). MathSciNetMATHCrossRef R. Ahlswede, A. Winter, Strong converse for identification via quantum channels, IEEE Trans. Inf. Theory 48(3), 569–579 (2002). MathSciNetMATHCrossRef
2.
Zurück zum Zitat E. Arias-Castro, Y.C. Eldar, Noise folding in compressed sensing, IEEE Signal Process. Lett. 18(8), 478–481 (2011). CrossRef E. Arias-Castro, Y.C. Eldar, Noise folding in compressed sensing, IEEE Signal Process. Lett. 18(8), 478–481 (2011). CrossRef
3.
Zurück zum Zitat R.G. Baraniuk, M. Davenport, R.A. DeVore, M. Wakin, A simple proof of the restricted isometry property for random matrices, Constr. Approx. 28(3), 253–263 (2008). MathSciNetMATHCrossRef R.G. Baraniuk, M. Davenport, R.A. DeVore, M. Wakin, A simple proof of the restricted isometry property for random matrices, Constr. Approx. 28(3), 253–263 (2008). MathSciNetMATHCrossRef
4.
Zurück zum Zitat H. Bauschke, H. Borwein, On projection algorithms for solving convex feasibility problems, SIAM Rev. 38(3), 367–426 (1996). MathSciNetMATHCrossRef H. Bauschke, H. Borwein, On projection algorithms for solving convex feasibility problems, SIAM Rev. 38(3), 367–426 (1996). MathSciNetMATHCrossRef
6.
Zurück zum Zitat E.J. Candès, Ridgelets: estimating with ridge functions, Ann. Stat. 31(5), 1561–1599 (2003). MATHCrossRef E.J. Candès, Ridgelets: estimating with ridge functions, Ann. Stat. 31(5), 1561–1599 (2003). MATHCrossRef
7.
Zurück zum Zitat E.J. Candès, D.L. Donoho, Ridgelets: a key to higher-dimensional intermittency? Philos. Trans. R. Soc., Math. Phys. Eng. Sci. 357(1760), 2495–2509 (1999). MATHCrossRef E.J. Candès, D.L. Donoho, Ridgelets: a key to higher-dimensional intermittency? Philos. Trans. R. Soc., Math. Phys. Eng. Sci. 357(1760), 2495–2509 (1999). MATHCrossRef
8.
Zurück zum Zitat E.J. Candès, J. Romberg, T. Tao, Stable signal recovery from incomplete and inaccurate measurements, Commun. Pure Appl. Math. 59(8), 1207–1223 (2006). MATHCrossRef E.J. Candès, J. Romberg, T. Tao, Stable signal recovery from incomplete and inaccurate measurements, Commun. Pure Appl. Math. 59(8), 1207–1223 (2006). MATHCrossRef
9.
Zurück zum Zitat E.J. Candès, T. Tao, The Dantzig selector: statistical estimation when p is much larger than n, Ann. Stat. 35(6), 2313–2351 (2007). MATHCrossRef E.J. Candès, T. Tao, The Dantzig selector: statistical estimation when p is much larger than n, Ann. Stat. 35(6), 2313–2351 (2007). MATHCrossRef
10.
Zurück zum Zitat A. Cohen, W. Dahmen, R.A. DeVore, Compressed sensing and best k-term approximation, J. Am. Math. Soc. 22(1), 211–231 (2009). MathSciNetMATHCrossRef A. Cohen, W. Dahmen, R.A. DeVore, Compressed sensing and best k-term approximation, J. Am. Math. Soc. 22(1), 211–231 (2009). MathSciNetMATHCrossRef
11.
Zurück zum Zitat A. Cohen, I. Daubechies, R.A. DeVore, G. Kerkyacharian, D. Picard, Capturing ridge functions in high dimensions from point queries, Constr. Approx. 35(2), 225–243 (2012). CrossRef A. Cohen, I. Daubechies, R.A. DeVore, G. Kerkyacharian, D. Picard, Capturing ridge functions in high dimensions from point queries, Constr. Approx. 35(2), 225–243 (2012). CrossRef
12.
Zurück zum Zitat R. Courant, D. Hilbert, Methods of Mathematical Physics, II (Interscience, New York, 1962). R. Courant, D. Hilbert, Methods of Mathematical Physics, II (Interscience, New York, 1962).
13.
Zurück zum Zitat R.A. DeVore, G. Petrova, P. Wojtaszczyk, Instance optimality in probability with an ℓ 1-minimization decoder, Appl. Comput. Harmon. Anal. 27(3), 275–288 (2009). MathSciNetMATHCrossRef R.A. DeVore, G. Petrova, P. Wojtaszczyk, Instance optimality in probability with an 1-minimization decoder, Appl. Comput. Harmon. Anal. 27(3), 275–288 (2009). MathSciNetMATHCrossRef
15.
Zurück zum Zitat M. Fazel, Matrix rank minimization with applications, Ph.D. thesis, Stanford University, Palo Alto, CA, 2002. M. Fazel, Matrix rank minimization with applications, Ph.D. thesis, Stanford University, Palo Alto, CA, 2002.
16.
Zurück zum Zitat M. Fornasier, Numerical methods for sparse recovery, in Theoretical Foundations and Numerical Methods for Sparse Recovery, ed. by M. Fornasier, Radon Series on Computational and Applied Mathematics (De Gruyter, Berlin, 2010). CrossRef M. Fornasier, Numerical methods for sparse recovery, in Theoretical Foundations and Numerical Methods for Sparse Recovery, ed. by M. Fornasier, Radon Series on Computational and Applied Mathematics (De Gruyter, Berlin, 2010). CrossRef
17.
Zurück zum Zitat M. Fornasier, H. Rauhut, Compressive sensing, in Handbook of Mathematical Methods in Imaging, vol. 1, ed. by O. Scherzer (Springer, Berlin, 2010), pp. 187–229. M. Fornasier, H. Rauhut, Compressive sensing, in Handbook of Mathematical Methods in Imaging, vol. 1, ed. by O. Scherzer (Springer, Berlin, 2010), pp. 187–229.
18.
Zurück zum Zitat S. Foucart, A note on ensuring sparse recovery via ℓ 1-minimization, Appl. Comput. Harmon. Anal. 29(1), 97–103 (2010). MathSciNetMATHCrossRef S. Foucart, A note on ensuring sparse recovery via 1-minimization, Appl. Comput. Harmon. Anal. 29(1), 97–103 (2010). MathSciNetMATHCrossRef
19.
Zurück zum Zitat G. Golub, C.F. van Loan, Matrix Computations, 3rd edn. (Johns Hopkins University Press, Baltimore, 1996). MATH G. Golub, C.F. van Loan, Matrix Computations, 3rd edn. (Johns Hopkins University Press, Baltimore, 1996). MATH
20.
Zurück zum Zitat F. John, Plane Waves and Spherical Means Applied to Partial Differential Equations (Interscience, New York, 1955). MATH F. John, Plane Waves and Spherical Means Applied to Partial Differential Equations (Interscience, New York, 1955). MATH
21.
Zurück zum Zitat M. Ledoux, The Concentration of Measure Phenomenon (American Mathematical Society, Providence, 2001). MATH M. Ledoux, The Concentration of Measure Phenomenon (American Mathematical Society, Providence, 2001). MATH
22.
23.
Zurück zum Zitat E. Novak, H. Woźniakowski, Tractability of Multivariate Problems, Volume I: Linear Information, EMS Tracts in Mathematics, vol. 6 (Eur. Math. Soc., Zürich, 2008). MATHCrossRef E. Novak, H. Woźniakowski, Tractability of Multivariate Problems, Volume I: Linear Information, EMS Tracts in Mathematics, vol. 6 (Eur. Math. Soc., Zürich, 2008). MATHCrossRef
24.
Zurück zum Zitat E. Novak, H. Woźniakowski, Approximation of infinitely differentiable multivariate functions is intractable, J. Complex. 25, 398–404 (2009). MATHCrossRef E. Novak, H. Woźniakowski, Approximation of infinitely differentiable multivariate functions is intractable, J. Complex. 25, 398–404 (2009). MATHCrossRef
25.
Zurück zum Zitat R.I. Oliveira, Sums of random Hermitian matrices and an inequality by Rudelson, Electron. Commun. Probab. 15, 203–212 (2010). MathSciNetMATHCrossRef R.I. Oliveira, Sums of random Hermitian matrices and an inequality by Rudelson, Electron. Commun. Probab. 15, 203–212 (2010). MathSciNetMATHCrossRef
26.
Zurück zum Zitat S. Oymak, K. Mohan, M. Fazel, B. Hassibi, A simplified approach to recovery conditions for low-rank matrices, in Proc. Intl. Symp. Information Theory (ISIT) (2011). S. Oymak, K. Mohan, M. Fazel, B. Hassibi, A simplified approach to recovery conditions for low-rank matrices, in Proc. Intl. Symp. Information Theory (ISIT) (2011).
27.
28.
Zurück zum Zitat B. Recht, M. Fazel, P. Parillo, Guaranteed minimum rank solutions to linear matrix equations via nuclear norm minimization, SIAM Rev. 52(3), 471–501 (2010). MathSciNetMATHCrossRef B. Recht, M. Fazel, P. Parillo, Guaranteed minimum rank solutions to linear matrix equations via nuclear norm minimization, SIAM Rev. 52(3), 471–501 (2010). MathSciNetMATHCrossRef
29.
Zurück zum Zitat M. Rudelson, R. Vershynin, Sampling from large matrices: an approach through geometric functional analysis, J. ACM 54(4), 21 (2007) 19 pp. MathSciNetCrossRef M. Rudelson, R. Vershynin, Sampling from large matrices: an approach through geometric functional analysis, J. ACM 54(4), 21 (2007) 19 pp. MathSciNetCrossRef
30.
Zurück zum Zitat W. Rudin, Function Theory in the Unit Ball of ℂ n (Springer, New York, 1980). CrossRef W. Rudin, Function Theory in the Unit Ball of n (Springer, New York, 1980). CrossRef
31.
Zurück zum Zitat K. Schnass, J. Vybíral, Compressed learning of high-dimensional sparse functions, in Proc. ICASSP11 (2011). K. Schnass, J. Vybíral, Compressed learning of high-dimensional sparse functions, in Proc. ICASSP11 (2011).
32.
Zurück zum Zitat G.W. Stewart, Perturbation theory for the singular value decomposition, in SVD and Signal Processing, II, ed. by R.J. Vacarro (Amsterdam, Elsevier, 1991). G.W. Stewart, Perturbation theory for the singular value decomposition, in SVD and Signal Processing, II, ed. by R.J. Vacarro (Amsterdam, Elsevier, 1991).
35.
Zurück zum Zitat H. Weyl, Das asymptotische Verteilungsgesetz der Eigenwerte linearer partieller Differentialgleichungen (mit einer Anwendung auf die Theorie der Hohlraumstrahlung), Math. Ann. 71, 441–479 (1912). MathSciNetMATHCrossRef H. Weyl, Das asymptotische Verteilungsgesetz der Eigenwerte linearer partieller Differentialgleichungen (mit einer Anwendung auf die Theorie der Hohlraumstrahlung), Math. Ann. 71, 441–479 (1912). MathSciNetMATHCrossRef
36.
Zurück zum Zitat P. Wojtaszczyk, ℓ 1 minimisation with noisy data, Preprint (2011). P. Wojtaszczyk, 1 minimisation with noisy data, Preprint (2011).
Metadaten
Titel
Learning Functions of Few Arbitrary Linear Parameters in High Dimensions
verfasst von
Massimo Fornasier
Karin Schnass
Jan Vybiral
Publikationsdatum
01.04.2012
Verlag
Springer-Verlag
Erschienen in
Foundations of Computational Mathematics / Ausgabe 2/2012
Print ISSN: 1615-3375
Elektronische ISSN: 1615-3383
DOI
https://doi.org/10.1007/s10208-012-9115-y

Premium Partner