Least Angle Regression in Orthogonal Case

LARS(least angle regression) is one of the sparse modeling methods. This article considered LARS under orthogonal design matrix, which we refer to as LARSO. In this article, we showed that LARSO reduces to a simple non-iterative algorithm that is a greedy procedure with shrinkage estimation. Based on this result, we found that LARSO is exactly equivalent with a soft-thresholding method in which a threshold level at the

k

th step is the (

k

 + 1)th largest value of the absolute values of the least squares estimators. For LARSO,

C

p

type model selection criterion can be derived. It is not only interpreted as a criterion for choosing the number of steps/coefficients in a regression problem but also regarded as a criterion for determining an optimal threshold level in LARSO-oriented soft-thresholding method which may be useful especially in non-parametric regression problems. Furthermore, in the context of orthogonal non-parametric regression, we clarified relationship between LARSO with

C

p

type criterion and several methods such as the universal thresholding and SUREshrink in wavelet denoising.