Prediction in abundant high-dimensional linear regression. (English) Zbl 1279.62140
Summary: An abundant regression is one in which most of the predictors contribute information about the response, which is contrary to the common notion of a sparse regression where few of the predictors are relevant. We discuss asymptotic characteristics of the methodology for prediction in abundant linear regressions as the sample size and number of predictors increase in various alignments. We show that some of the estimators can perform well for the purpose of prediction in abundant high-dimensional regressions.
References:
| [1] | Cai, T. T., Liu, W. and Luo, X. (2011). A constrained \(\ell_{1}\) minimization approach to sparse precision matrix estimation. J. Amer. Statist. Assoc. 106 594-607. · Zbl 1232.62087 · doi:10.1198/jasa.2011.tm10155 |
| [2] | Christensen, R. (1987). Plane Answers to Complex Questions . Wiley, New York. · Zbl 0645.62076 |
| [3] | Chung, H. and Keleş, S. (2010). Sparse partial least squares regression for simultaneous dimension reduction and variable selection. Journal of The Royal Statistical Society Series B 72 3-25. |
| [4] | Cook, R. D. (2007). Fisher lecture: dimension reduction in regression. Statist. Sci. 22 1-26. · Zbl 1246.62148 |
| [5] | Cook, R. D. and Forzani, L. (2008). Principal fitted components for dimension reduction in regression. Statist. Sci. 23 485-501. · Zbl 1329.62274 |
| [6] | Cook, R. D. and Forzani, L. (2011). On the mean and variance of the generalized inverse of a singular Wishart matrix. Electronic Journal of Statistics 5 146-158. · Zbl 1274.62350 |
| [7] | Cook, R. D., Forzani, L. and Rothman, A. J. (2012). Estimating sufficient reductions of the predictors in abundant high-dimensional regressions. Ann. Statist. 40 353-384. · Zbl 1246.62150 |
| [8] | Dicker, L. (2012). Dense signals, linear estimators, and out-of-sample predictions for high-dimensional linear models. |
| [9] | Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348-1360. · Zbl 1073.62547 · doi:10.1198/016214501753382273 |
| [10] | Frank, I. E. and Friedman, J. H. (1993). A statistical view of some chemometrics regression tools. Technometrics 35 109-135. · Zbl 0775.62288 · doi:10.2307/1269656 |
| [11] | Friedman, J., Hastie, T. and Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9 432-441. · Zbl 1143.62076 · doi:10.1093/biostatistics/kxm045 |
| [12] | Henmi, M. and Eguchi, S. (2004). A paradox converning nuisance parameters and projected estimating functions. Biometrika 91 929-941. · Zbl 1064.62002 · doi:10.1093/biomet/91.4.929 |
| [13] | Hoerl, A. E. and Kennard, R. W. (1970). Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12 55-67. · Zbl 0202.17205 · doi:10.2307/1267351 |
| [14] | Hsieh, C.-J., Sustik, M. A., Dhillon, I. S. and Ravikumar, P. K. (2011). Sparse inverse covariance matrix estimation using quadratic approximation. In Advances in Neural Information Processing Systems , 24 2330-2338. MIT Press, Cambridge, MA. |
| [15] | Jeng, X. J. and Daye, Z. J. (2011). Sparse covariance thresholding for high-dimensional variable selection. Statistica Sinica 21 625-657. · Zbl 1214.62059 · doi:10.5705/ss.2011.028a |
| [16] | Letac, G. and Massan, H. (2004). All invariant moments of the Wishart distribution. Scand. J. Statist. 31 295-318. · Zbl 1063.62081 · doi:10.1111/j.1467-9469.2004.01-043.x |
| [17] | Magnus, J. R. and Neudecker, H. (1979). The commutation matrix: some properties and applications. Ann. Statist. 7 381-394. · Zbl 0414.62040 · doi:10.1214/aos/1176344621 |
| [18] | Muirhead, R. J. (1982). Aspects of Multivariate Statistical Theory . Wiley, New York. · Zbl 0556.62028 |
| [19] | Pourahmadi, M. (2011). Modeling covariance matrices: the GLM and regularization perspectives. Statistical Science 26 369-387. · Zbl 1246.62139 |
| [20] | Ravikumar, P., Wainwright, M. J., Raskutti, G. and Yu, B. (2011). High-dimensional covariance estimation by minimizing l1-penalized log-determinant divergence. Electronic Journal of Statistics 5 935-980. · Zbl 1274.62190 |
| [21] | Rothman, A. J., Bickel, P. J., Levina, E. and Zhu, J. (2008). Sparse permutation invariant covariance estimation. Electronic Journal of Statistics 2 494-515. · Zbl 1320.62135 |
| [22] | Sæbø, S., Almøy, T., Aarøe, J. and Aastveit, A. H. (2007). ST-PLS: a multi-directional nearest shrunken centroid type classifier via PLS. Journal of Chemometrics 20 54-62. |
| [23] | Shao, J. and Deng, X. (2012). Estimation in high-dimensional linear models with deterministic design matrices. Ann. Statist. 40 812-831. · Zbl 1273.62177 |
| [24] | Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc., Ser. B 58 267-288. · Zbl 0850.62538 |
| [25] | von Rosen, D. (1988). Moments of the inverted Wishart distribution. Scand. J. Statist. 15 97-109. · Zbl 0663.62063 |
| [26] | Witten, D. M. and Tibshirani, R. (2009). Covariance-regularized regression and classification for high-dimensional problems. Journal of The Royal Statistical Society Series B 71 615-636. · Zbl 1250.62033 |
| [27] | Yuan, M. and Lin, Y. (2007). Model selection and estimation in the Gaussian graphical model. Biometrika 94 19-35. · Zbl 1142.62408 · doi:10.1093/biomet/asm018 |
| [28] | Zou, H. (2005). Regularization and variable selection via the elastic net. Journal of The Royal Statistical Society Series B 67 301-320. · Zbl 1069.62054 · doi:10.1111/j.1467-9868.2005.00503.x |
| [29] | Zou, H. (2006). The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101 1418-1429. · Zbl 1171.62326 · doi:10.1198/016214506000000735 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
