High-dimensional inference: confidence intervals, \(p\)-values and R-software hdi. (English) Zbl 1426.62183
Summary: We present a (selective) review of recent frequentist high-dimensional inference methods for constructing \(p\)-values and confidence intervals in linear and generalized linear models. We include a broad, comparative empirical study which complements the viewpoint from statistical methodology and theory. Furthermore, we introduce and illustrate the R-package hdi which easily allows the use of different methods and supports reproducibility.
MSC:
| 62H30 | Classification and discrimination; cluster analysis (statistical aspects) |
| 62F25 | Parametric tolerance and confidence regions |
| 62J05 | Linear regression; mixed models |
| 62J12 | Generalized linear models (logistic models) |
| 62J15 | Paired and multiple comparisons; multiple testing |
| 62-04 | Software, source code, etc. for problems pertaining to statistics |
Keywords:
clustering; confidence interval; generalized linear model; high-dimensional statistical inference; linear model; multiple testing; \(p\)-valueReferences:
| [1] | Barber, R. F. and Candès, E. J. (2015). Controlling the false discovery rate via knockoffs. Ann. Statist.43 2055-2085. · Zbl 1327.62082 · doi:10.1214/15-AOS1337 |
| [2] | Belloni, A., Chernozhukov, V. and Kato, K. (2015). Uniform post-selection inference for least absolute deviation regression and other \(Z\)-estimation problems. Biometrika102 77-94. · Zbl 1345.62049 · doi:10.1093/biomet/asu056 |
| [3] | Belloni, A., Chernozhukov, V. and Wang, L. (2011). Square-root Lasso: Pivotal recovery of sparse signals via conic programming. Biometrika98 791-806. · Zbl 1228.62083 · doi:10.1093/biomet/asr043 |
| [4] | Belloni, A., Chen, D., Chernozhukov, V. and Hansen, C. (2012). Sparse models and methods for optimal instruments with an application to eminent domain. Econometrica80 2369-2429. · Zbl 1274.62464 · doi:10.3982/ECTA9626 |
| [5] | Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Ann. Statist.29 1165-1188. · Zbl 1041.62061 · doi:10.1214/aos/1013699998 |
| [6] | Benjamini, Y. and Yekutieli, D. (2005). False discovery rate-adjusted multiple confidence intervals for selected parameters. J. Amer. Statist. Assoc.100 71-93. · Zbl 1117.62302 · doi:10.1198/016214504000001907 |
| [7] | Berk, R., Brown, L., Buja, A., Zhang, K. and Zhao, L. (2013). Valid post-selection inference. Ann. Statist.41 802-837. · Zbl 1267.62080 · doi:10.1214/12-AOS1077 |
| [8] | Bogdan, M., van den Berg, E., Su, W. and Candès, E. (2013). Statistical estimation and testing via the sorted l1 norm. Preprint. Available at arXiv:1310.1969. |
| [9] | Bogdan, M., van den Berg, E., Sabatti, C., Su, W. and Candès, E. (2014). SLOPE—adaptive variable selection via convex optimization. Preprint. Available at arXiv:1407.3824. · Zbl 1454.62212 |
| [10] | Breiman, L. (1996a). Bagging predictors. Mach. Learn.24 123-140. · Zbl 0858.68080 |
| [11] | Breiman, L. (1996b). Heuristics of instability and stabilization in model selection. Ann. Statist.24 2350-2383. · Zbl 0867.62055 · doi:10.1214/aos/1032181158 |
| [12] | Bühlmann, P. (2013). Statistical significance in high-dimensional linear models. Bernoulli19 1212-1242. · Zbl 1273.62173 · doi:10.3150/12-BEJSP11 |
| [13] | Bühlmann, P., Kalisch, M. and Meier, L. (2014). High-dimensional statistics with a view towards applications in biology. Annual Review of Statistics and Its Applications1 255-278. |
| [14] | Bühlmann, P. and Mandozzi, J. (2014). High-dimensional variable screening and bias in subsequent inference, with an empirical comparison. Comput. Statist.29 407-430. · Zbl 1306.65035 · doi:10.1007/s00180-013-0436-3 |
| [15] | Bühlmann, P., Meier, L. and van de Geer, S. (2014). Discussion: “A significance test for the Lasso”. Ann. Statist.42 469-477. · Zbl 1305.62248 · doi:10.1214/13-AOS1175A |
| [16] | Bühlmann, P. and van de Geer, S. (2011). Statistics for High-Dimensional Data: Methods, Theory and Applications. Springer, Heidelberg. · Zbl 1273.62015 |
| [17] | Bühlmann, P. and van de Geer, S. (2015). High-dimensional inference in misspecified linear models. Electron. J. Stat.9 1449-1473. · Zbl 1327.62420 · doi:10.1214/15-EJS1041 |
| [18] | Candes, E. J. and Tao, T. (2006). Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans. Inform. Theory52 5406-5425. · Zbl 1309.94033 · doi:10.1109/TIT.2006.885507 |
| [19] | Chandrasekaran, V., Parrilo, P. A. and Willsky, A. S. (2012). Latent variable graphical model selection via convex optimization. Ann. Statist.40 1935-1967. · Zbl 1257.62061 · doi:10.1214/11-AOS949 |
| [20] | Chatterjee, A. and Lahiri, S. N. (2013). Rates of convergence of the adaptive LASSO estimators to the oracle distribution and higher order refinements by the bootstrap. Ann. Statist.41 1232-1259. · Zbl 1293.62153 · doi:10.1214/13-AOS1106 |
| [21] | Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B. Stat. Methodol.39 1-38. · Zbl 0364.62022 |
| [22] | Dezeure, R., Bühlmann, P., Meier, L. and Meinshausen, N. (2015). Supplement to “High-Dimensional Inference: Confidence Intervals, \(p\)-Values and R-Software hdi.” DOI:10.1214/15-STS527SUPP. · Zbl 1426.62183 |
| [23] | Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc.96 1348-1360. · Zbl 1073.62547 · doi:10.1198/016214501753382273 |
| [24] | Fan, J. and Lv, J. (2008). Sure independence screening for ultrahigh dimensional feature space. J. R. Stat. Soc. Ser. B. Stat. Methodol.70 849-911. · Zbl 1411.62187 · doi:10.1111/j.1467-9868.2008.00674.x |
| [25] | Fan, J. and Lv, J. (2010). A selective overview of variable selection in high dimensional feature space. Statist. Sinica20 101-148. · Zbl 1180.62080 |
| [26] | Fan, J., Xue, L. and Zou, H. (2014). Strong oracle optimality of folded concave penalized estimation. Ann. Statist.42 819-849. · Zbl 1305.62252 · doi:10.1214/13-AOS1198 |
| [27] | Fithian, W., Sun, D. and Taylor, J. (2014). Optimal inference after model selection. Preprint. Available at arXiv:1410.2597. |
| [28] | Hartigan, J. A. (1975). Clustering Algorithms. Wiley, New York. · Zbl 0372.62040 |
| [29] | Hastie, T., Tibshirani, R. and Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. Springer, New York. · Zbl 1273.62005 |
| [30] | Javanmard, A. and Montanari, A. (2014). Confidence intervals and hypothesis testing for high-dimensional regression. J. Mach. Learn. Res.15 2869-2909. · Zbl 1319.62145 |
| [31] | Knight, K. and Fu, W. (2000). Asymptotics for Lasso-type estimators. Ann. Statist.28 1356-1378. · Zbl 1105.62357 · doi:10.1214/aos/1015957397 |
| [32] | Lee, J., Sun, D., Sun, Y. and Taylor, J. (2013). Exact post-selection inference, with application to the Lasso. Preprint. Available at arXiv:1311.6238. · Zbl 1341.62061 |
| [33] | Leeb, H. and Pötscher, B. M. (2003). The finite-sample distribution of post-model-selection estimators and uniform versus nonuniform approximations. Econometric Theory19 100-142. · Zbl 1032.62011 · doi:10.1017/S0266466603191050 |
| [34] | Liu, H. and Yu, B. (2013). Asymptotic properties of Lasso \({}+{}\) mLS and Lasso \({}+{}\) Ridge in sparse high-dimensional linear regression. Electron. J. Stat.7 3124-3169. · Zbl 1281.62158 · doi:10.1214/14-EJS875 |
| [35] | Lockhart, R., Taylor, J., Tibshirani, R. J. and Tibshirani, R. (2014). A significance test for the Lasso. Ann. Statist.42 413-468. · Zbl 1305.62254 · doi:10.1214/13-AOS1175 |
| [36] | Mandozzi, J. and Bühlmann, P. (2015). Hierarchical testing in the high-dimensional setting with correlated variables. J. Amer. Statist. Assoc. To appear. DOI:10.1080/01621459.2015.1007209. Available at arXiv:1312.5556. · Zbl 1306.65035 |
| [37] | McCullagh, P. and Nelder, J. A. (1983). Generalized Linear Models, 2nd ed. Chapman & Hall, London. · Zbl 0588.62104 |
| [38] | Meinshausen, N. (2008). Hierarchical testing of variable importance. Biometrika95 265-278. · Zbl 1437.62557 · doi:10.1093/biomet/asn007 |
| [39] | Meinshausen, N. (2015). Group-bound: Confidence intervals for groups of variables in sparse high-dimensional regression without assumptions on the design. J. R. Stat. Soc. Ser. B. Stat. Methodol. To appear. DOI:10.1111/rssb.12094. Available at arXiv:1309.3489. · Zbl 1414.62060 |
| [40] | Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the Lasso. Ann. Statist.34 1436-1462. · Zbl 1113.62082 · doi:10.1214/009053606000000281 |
| [41] | Meinshausen, N. and Bühlmann, P. (2010). Stability selection. J. R. Stat. Soc. Ser. B. Stat. Methodol.72 417-473. · Zbl 1411.62142 · doi:10.1111/j.1467-9868.2010.00740.x |
| [42] | Meinshausen, N., Meier, L. and Bühlmann, P. (2009). \(p\)-values for high-dimensional regression. J. Amer. Statist. Assoc.104 1671-1681. · Zbl 1205.62089 |
| [43] | Pearl, J. (2000). Causality: Models, Reasoning, and Inference. Cambridge Univ. Press, Cambridge. · Zbl 0959.68116 |
| [44] | Reid, S., Tibshirani, R. and Friedman, J. (2013). A study of error variance estimation in Lasso regression. Preprint. Available at arXiv:1311.5274. · Zbl 1372.62023 |
| [45] | Shah, R. D. and Samworth, R. J. (2013). Variable selection with error control: Another look at stability selection. J. R. Stat. Soc. Ser. B. Stat. Methodol.75 55-80. · Zbl 07555438 · doi:10.1111/j.1467-9868.2011.01034.x |
| [46] | Shao, J. and Deng, X. (2012). Estimation in high-dimensional linear models with deterministic design matrices. Ann. Statist.40 812-831. · Zbl 1273.62177 · doi:10.1214/12-AOS982 |
| [47] | Spirtes, P., Glymour, C. and Scheines, R. (2000). Causation, Prediction, and Search, 2nd ed. MIT Press, Cambridge, MA. · Zbl 0806.62001 |
| [48] | Sun, T. and Zhang, C.-H. (2012). Scaled sparse linear regression. Biometrika99 879-898. · Zbl 1452.62515 · doi:10.1093/biomet/ass043 |
| [49] | Taylor, J., Lockhart, R., Tibshirani, R. and Tibshirani, R. (2014). Exact post-selection inference for forward stepwise and least angle regression. Preprint. Available at arXiv:1401.3889. · Zbl 1305.62254 |
| [50] | Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. J. R. Stat. Soc. Ser. B. Stat. Methodol.58 267-288. · Zbl 0850.62538 |
| [51] | van de Geer, S. (2007). The deterministic Lasso. In JSM Proceedings 140. American Statistical Association, Alexandria, VA. |
| [52] | van de Geer, S. (2014). Statistical theory for high-dimensional models. Preprint. Available at arXiv:1409.8557. · Zbl 1430.62162 |
| [53] | van de Geer, S. (2015). \(χ^2\)-confidence sets in high-dimensional regression. Preprint. Available at arXiv:1502.07131. |
| [54] | van de Geer, S. A. and Bühlmann, P. (2009). On the conditions used to prove oracle results for the Lasso. Electron. J. Stat.3 1360-1392. · Zbl 1327.62425 · doi:10.1214/09-EJS506 |
| [55] | van de Geer, S., Bühlmann, P. and Zhou, S. (2011). The adaptive and the thresholded Lasso for potentially misspecified models (and a lower bound for the Lasso). Electron. J. Stat.5 688-749. · Zbl 1274.62471 · doi:10.1214/11-EJS624 |
| [56] | van de Geer, S., Bühlmann, P., Ritov, Y. and Dezeure, R. (2014). On asymptotically optimal confidence regions and tests for high-dimensional models. Ann. Statist.42 1166-1202. · Zbl 1305.62259 · doi:10.1214/14-AOS1221 |
| [57] | Wasserman, L. (2014). Discussion: “A significance test for the Lasso”. Ann. Statist.42 501-508. · Zbl 1305.62257 · doi:10.1214/13-AOS1175E |
| [58] | Wasserman, L. and Roeder, K. (2009). High-dimensional variable selection. Ann. Statist.37 2178-2201. · Zbl 1173.62054 · doi:10.1214/08-AOS646 |
| [59] | Yuan, M. and Lin, Y. (2006). Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B. Stat. Methodol.68 49-67. · Zbl 1141.62030 · doi:10.1111/j.1467-9868.2005.00532.x |
| [60] | Zhang, C.-H. (2010). Nearly unbiased variable selection under minimax concave penalty. Ann. Statist.38 894-942. · Zbl 1183.62120 · doi:10.1214/09-AOS729 |
| [61] | Zhang, C.-H. and Zhang, S. S. (2014). Confidence intervals for low dimensional parameters in high dimensional linear models. J. R. Stat. Soc. Ser. B. Stat. Methodol.76 217-242. · Zbl 1411.62196 · doi:10.1111/rssb.12026 |
| [62] | Zou, H. (2006). The adaptive Lasso and its oracle properties. J. Amer. Statist. Assoc.101 1418-1429. · Zbl 1171.62326 · doi:10.1198/016214506000000735 |
| [63] | Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B. Stat. Methodol.67 301-320. · Zbl 1069.62054 · doi:10.1111/j.1467-9868.2005.00503.x |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
