Top

Published in:

2018 | OriginalPaper | Chapter

10. Analysis of High-Dimensional Regression Models Using Orthogonal Greedy Algorithms

Authors : Hsiang-Ling Hsu, Ching-Kang Ing, Tze Leung Lai

Published in: Handbook of Big Data Analytics

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

We begin by reviewing recent results of Ing and Lai (Stat Sin 21:1473–1513, 2011) on the statistical properties of the orthogonal greedy algorithm (OGA) in high-dimensional sparse regression models with independent observations. In particular, when the regression coefficients are absolutely summable, the conditional mean squared prediction error and the empirical norm of OGA derived by Ing and Lai (Stat Sin 21:1473–1513, 2011) are introduced. We then explore the performance of OGA under more general sparsity conditions. Finally, we obtain the convergence rate of OGA in high-dimensional time series models, and illustrate the advantage of our results compared to those established for Lasso by Basu and Michailidis (Ann Stat 43:1535–1567, 2015) and Wu and Wu (Electron J Stat 10:352–379, 2016).

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter High-Dimensional Classification

next chapter Semi-supervised Smoothing for Large Data Problems

Basu S, Michailidis G (2015) Regularized estimation in sparse high-dimensional time series models. Ann Stat 43:1535–1567MathSciNetCrossRef

Bickel PJ, Levina E (2008) Regularized estimation of large covariance matrices. Ann Stat 36: 199–227MathSciNetCrossRef

Bickel PJ, Ritov Y, Tsybakov AB (2009) Simultaneous analysis of Lasso and Dantzig selector. Ann Stat 37:1705–1732MathSciNetCrossRef

Bühlmann P (2006) Boosting for high-dimensional linear models. Ann Stat 34:559–583MathSciNetCrossRef

Bunea F, Tsybakov AB, Wegkamp MH (2007) Sparsity oracle inequalities for the Lasso. Electr J Stat 1:169–194MathSciNetCrossRef

Candés EJ, Plan Y (2009) Near-ideal model selection by ℓ ₁ minimization. Ann Stat 37:2145–2177MathSciNetCrossRef

Candés EJ, Tao T (2007) The Dantzig selector: statistical estimation when p is much larger than n. Ann Stat 35:2313–2351MathSciNetCrossRef

Cai T, Zhang C-H, Zhou HH (2010) Optimal rates of convergence for covariance matrix estimation. Ann Stat 38:2118–2144MathSciNetCrossRef

Donoho DL, Elad M, Temlyakov VN (2006) Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Trans Inform Theory 52:6–18MathSciNetCrossRef

Fan J, Li R (2001) Variable selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc 96:1348–1360MathSciNetCrossRef

Findley DF, Wei C-Z (1993) Moment bounds for deriving time series CLT’s and model selection procedures. Stat Sin 3:453–470

Foster DP, George EI (1994) The risk inflation criterion for multiple regression. Ann Stat 22:1947–1975MathSciNetCrossRef

Friedman J, Hastie T, Tibshirani R (2010) glmnet: Lasso and elastic-net regularized generalized linear models. R package version 1.1-5. http://cran.r-project.org/web/packages/glmnet/index.html. Accessed 10 Dec 2012

Gao F, Ing C-K, Yang Y (2013) Metric entropy and sparse linear approximation of l _q-Hulls for 0 < q ≤ 1. J Approx Theory 166:42–55

Ing C-K, Lai TL (2011) A stepwise regression method and consistent model selection for high-dimensional sparse linear models. Stat Sin 21:1473–1513

Ing C-K, Lai TL (2015) An efficient pathwise variable selection criterion in weakly sparse regression models. Technical Report, Academia Sinica

Ing C-K, Lai TL (2016) Model selection for high-dimensional time series. Technical Report, Academia Sinica

Ing C-K, Wei C-Z (2003) On same-realization prediction in an infinite-order autoregressive process. J Multivar Anal 85:130–155MathSciNetCrossRef

Negahban SN, Ravikumar P, Wainwright MJ, Yu B (2012) A unified framework for high-dimensional analysis of m-estimators with decomposable regularizers. Stat Sci 27:538–557MathSciNetCrossRef

Raskutti G, Wainwright MJ, Yu B (2011) Minimax rates of estimation for high-dimensional linear regression over l _q-balls. IEEE Trans Inform Theory 57:6976–6994MathSciNetCrossRef

Temlyakov VN (2000) Weak greedy algorithms. Adv Comput Math 12:213–227MathSciNetCrossRef

Tibshirani R (1996) Regression shrinkage and selection via the Lasso. J R Stat Soc Ser B 58:267–288MathSciNetMATH

Tropp JA (2004) Greed is good: algorithmic results for sparse approximation. IEEE Trans Inform Theory 50:2231–2242MathSciNetCrossRef

Wang Z, Paterlini S, Gao F, Yang Y (2014) Adaptive minimax regression estimation over sparse hulls. J Mach Learn Res 15:1675–1711MathSciNetMATH

Wei C-Z (1987) Adaptive prediction by least squares predictors in stochastic regression models with applications to time series. Ann Stat 15:1667–1682MathSciNetCrossRef

Wu WB, Wu YN (2016) Performance bounds for parameter estimates of high-dimensional linear models with correlated errors. Electron J Stat 10:352–379MathSciNetCrossRef

Zhang C-H, Huang J (2008) The sparsity and bias of the Lasso selection in highdimensional linear regression. Ann Stat 36:1567–1594CrossRef

Zhao P, Yu B (2006) On model selection consistency of Lasso. J Mach Learn Res 7:2541–2563MathSciNetMATH

Zou H (2006) The adaptive Lasso and its oracle properties. J Am Stat Assoc 101:1418–1429MathSciNetCrossRef

Title: Analysis of High-Dimensional Regression Models Using Orthogonal Greedy Algorithms
Authors: Hsiang-Ling Hsu
Ching-Kang Ing
Tze Leung Lai
Publisher: Springer International Publishing
Book: Handbook of Big Data Analytics
Print ISBN: 978-3-319-18283-4

Electronic ISBN: 978-3-319-18284-1

Copyright Year: 2018
DOI: https://doi.org/10.1007/978-3-319-18284-1_10

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner