Distributed testing and estimation under sparse high dimensional models. (English) Zbl 1392.62060
Summary: This paper studies hypothesis testing and parameter estimation in the context of the divide-and-conquer algorithm. In a unified likelihood-based framework, we propose new test statistics and point estimators obtained by aggregating various statistics from \(k\) subsamples of size \(n/k\), where \(n\) is the sample size. In both low dimensional and sparse high dimensional settings, we address the important question of how large \(k\) can be, as \(n\) grows large, such that the loss of efficiency due to the divide-and-conquer algorithm is negligible. In other words, the resulting estimators have the same inferential efficiencies and estimation rates as an oracle with access to the full sample. Thorough numerical results are provided to back up the theory.
MSC:
| 62F05 | Asymptotic properties of parametric tests |
| 62F10 | Point estimation |
| 62F12 | Asymptotic properties of parametric estimators |
References:
| [1] | Battey, H., Fan, J., Liu, H., Lu, J. and Zhu, Z. (2018). Supplement to “Distributed testing and estimation under sparse high dimensional models.” DOI:10.1214/17-AOS1587SUPP. · Zbl 1392.62060 |
| [2] | Bickel, P. J. (1975). One-step Huber estimates in the linear model. J. Amer. Statist. Assoc.70 428-434. · Zbl 0322.62038 · doi:10.1080/01621459.1975.10479884 |
| [3] | Bühlmann, P. and van de Geer, S. (2011). Statistics for High-Dimensional Data: Methods, Theory and Applications. Springer, Heidelberg. · Zbl 1273.62015 |
| [4] | Candes, E. and Tao, T. (2007). The Dantzig selector: Statistical estimation when \(p\) is much larger than \(n\). Ann. Statist.35 2313-2351. · Zbl 1139.62019 · doi:10.1214/009053606000001523 |
| [5] | Chen, X. and Xie, M. (2014). A split-and-conquer approach for analysis of extraordinarily large data. Statist. Sinica 24 1655-1684. · Zbl 1480.62258 |
| [6] | Chernozhukov, V., Chetverikov, D. and Kato, K. (2013). Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors. Ann. Statist.41 2786-2819. · Zbl 1292.62030 · doi:10.1214/13-AOS1161 |
| [7] | Cox, D. R. and Hinkley, D. V. (1974). Theoretical Statistics. Chapman & Hall, London. · Zbl 0334.62003 |
| [8] | Fan, J., Guo, S. and Hao, N. (2012). Variance estimation using refitted cross-validation in ultrahigh dimensional regression. J. R. Stat. Soc. Ser. B. Stat. Methodol.74 37-65. · Zbl 1411.62199 |
| [9] | Fan, J., Han, F. and Liu, H. (2014). Challenges of big data analysis. Nat. Sci. Rev.1 293-314. |
| [10] | Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc.96 1348-1360. · Zbl 1073.62547 · doi:10.1198/016214501753382273 |
| [11] | Fan, J. and Lv, J. (2011). Nonconcave penalized likelihood with NP-dimensionality. IEEE Trans. Inform. Theory 57 5467-5484. · Zbl 1365.62277 · doi:10.1109/TIT.2011.2158486 |
| [12] | Fan, J. and Song, R. (2010). Sure independence screening in generalized linear models with NP-dimensionality. Ann. Statist.38 3567-3604. · Zbl 1206.68157 · doi:10.1214/10-AOS798 |
| [13] | Javanmard, A. and Montanari, A. (2014). Confidence intervals and hypothesis testing for high-dimensional regression. J. Mach. Learn. Res.15 2869-2909. · Zbl 1319.62145 |
| [14] | Kallenberg, O. (1997). Foundations of Modern Probability. Springer, New York. · Zbl 0892.60001 |
| [15] | Kleiner, A., Talwalkar, A., Sarkar, P. and Jordan, M. I. (2014). A scalable bootstrap for massive data. J. R. Stat. Soc. Ser. B. Stat. Methodol.76 795-816. · Zbl 07555464 |
| [16] | Lee, J. D., Liu, Q., Sun, Y. and Taylor, J. E. (2017). Communication-efficient sparse regression. J. Mach. Learn. Res.18 Paper No. 5. · Zbl 1434.62157 |
| [17] | Liu, Q. and Ihler, A. T. (2014). Distributed estimation, information loss and exponential families. In Advances in Neural Information Processing Systems 27 (Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence and K. Q. Weinberger, eds.) 1098-1106. MIT Press, Cambridge, MA. |
| [18] | Loh, P.-L. and Wainwright, M. J. (2013). Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optima. In Advances in Neural Information Processing Systems 26 (C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani and K. Q. Weinberger, eds.) 476-484. Curran Associates Inc., Red Hook, NY. |
| [19] | Loh, P.-L. and Wainwright, M. J. (2015). Regularized \(M\)-estimators with nonconvexity: Statistical and algorithmic theory for local optima. J. Mach. Learn. Res.16 559-616. · Zbl 1360.62276 |
| [20] | Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. Ann. Statist.34 1436-1462. · Zbl 1113.62082 · doi:10.1214/009053606000000281 |
| [21] | Negahban, S. N., Yu, B., Wainwright, M. J. and Ravikumar, P. (2009). A unified framework for high-dimensional analysis of \(M\)-estimators with decomposable regularizers. In Advances in Neural Information Processing Systems 22 (Y. Bengio, D. Schuurmans, J. D. Lafferty, C. K. I. Williams and A. Culotta, eds.) 1348-1356. Curran Associates Inc., Red Hook, NY. · Zbl 1331.62350 · doi:10.1214/12-STS400 |
| [22] | Ning, Y. and Liu, H. (2017). A general theory of hypothesis tests and confidence regions for sparse high dimensional models. Ann. Statist.45 158-195. · Zbl 1364.62128 · doi:10.1214/16-AOS1448 |
| [23] | Rosenblatt, J. D. and Nadler, B. (2016). On the optimality of averaging in distributed statistical learning. Inf. Inference 5 379-404. · Zbl 1426.68241 · doi:10.1093/imaiai/iaw013 |
| [24] | Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B. Stat. Methodol.58 267-288. · Zbl 0850.62538 |
| [25] | van de Geer, S., Bühlmann, P., Ritov, Y. and Dezeure, R. (2014). On asymptotically optimal confidence regions and tests for high-dimensional models. Ann. Statist.42 1166-1202. · Zbl 1305.62259 · doi:10.1214/14-AOS1221 |
| [26] | Wang, Z., Liu, H. and Zhang, T. (2014). Optimal computational and statistical rates of convergence for sparse nonconvex learning problems. Ann. Statist.42 2164-2201. · Zbl 1302.62066 · doi:10.1214/14-AOS1238 |
| [27] | Zhang, C.-H. (2010). Nearly unbiased variable selection under minimax concave penalty. Ann. Statist.38 894-942. · Zbl 1183.62120 · doi:10.1214/09-AOS729 |
| [28] | Zhang, Y., Duchi, J. and Wainwright, M. (2015). Divide and conquer kernel ridge regression: A distributed algorithm with minimax optimal rates. J. Mach. Learn. Res.16 3299-3340. · Zbl 1351.62142 |
| [29] | Zhang, C.-H. and Zhang, T. (2012). A general theory of concave regularization for high-dimensional sparse estimation problems. Statist. Sci.27 576-593. · Zbl 1331.62353 · doi:10.1214/12-STS399 |
| [30] | Zhang, C.-H. and Zhang, S. S. (2014). Confidence intervals for low dimensional parameters in high dimensional linear models. J. R. Stat. Soc. Ser. B. Stat. Methodol.76 217-242. · Zbl 1411.62196 · doi:10.1111/rssb.12026 |
| [31] | Zhao, T., Cheng, G. and Liu, H. (2016). A partially linear framework for massive heterogeneous data. Ann. Statist.44 1400-1437. · Zbl 1358.62050 · doi:10.1214/15-AOS1410 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
