The Lasso, correlated design, and improved oracle inequalities. (English) Zbl 1327.62426
Banerjee, M. (ed.) et al., From probability to statistics and back: high-dimensional models and processes. A Festschrift in honor of Jon A. Wellner. Including papers from the conference, Seattle, WA, USA, July 28–31, 2010. Beachwood, OH: IMS, Institute of Mathematical Statistics (ISBN 978-0-940600-83-6). Institute of Mathematical Statistics Collections 9, 303-316 (2013).
Summary: We study high-dimensional linear models and the \(\ell_1\)-penalized least squares estimator, also known as the Lasso estimator. In literature, oracle inequalities have been derived under restricted eigenvalue or compatibility conditions. In this paper, we complement this with entropy conditions which allow one to improve the dual norm bound, and demonstrate how this leads to new oracle inequalities. The new oracle inequalities show that a smaller choice for the tuning parameter and a trade-off between \(\ell_1\)-norms and small compatibility constants are possible. This implies, in particular for correlated design, improved bounds for the prediction error of the Lasso estimator as compared to the methods based on restricted eigenvalue or compatibility conditions only.
For the entire collection see [Zbl 1319.62002].
For the entire collection see [Zbl 1319.62002].
MSC:
| 62J07 | Ridge regression; shrinkage estimators (Lasso) |
| 62J05 | Linear regression; mixed models |
| 62J99 | Linear inference, regression |
