Estimation of conditional mean operator under the bandable covariance structure. (English) Zbl 1493.62038
Summary: We consider high-dimensional multivariate linear regression models, where the joint distribution of covariates and response variables is a multivariate normal distribution with a bandable covariance matrix. The main goal of this paper is to estimate the regression coefficient matrix, which is a function of the bandable covariance matrix. Although the tapering estimator of covariance has the minimax optimal convergence rate for the class of bandable covariances, we show that it is sub-optimal for the regression coefficient; that is, a minimax estimator for the class of bandable covariances may not be a minimax estimator for its functionals. We propose the blockwise tapering estimator of the regression coefficient, which has the minimax optimal convergence rate for the regression coefficient under the bandable covariance assumption. We also propose a Bayesian procedure called the blockwise tapering post-processed posterior of the regression coefficient and show that the proposed Bayesian procedure has the minimax optimal convergence rate for the regression coefficient under the bandable covariance assumption. We show that the proposed methods outperform the existing methods via numerical studies.
MSC:
| 62C20 | Minimax procedures in statistical decision theory |
| 62H12 | Estimation in multivariate analysis |
| 62J05 | Linear regression; mixed models |
Keywords:
bandable covariance; conditional mean; covariance estimation; minimax analysis; post-processed posteriorSoftware:
SOFARReferences:
| [1] | Bashir, A., Carvalho, C. M., Hahn, P. R., Jones, M. B. et al. (2018). Post-processing posteriors over precision matrices to produce sparse graph estimates, Bayesian Analysis. · Zbl 1435.62103 |
| [2] | Bickel, P. J. and Levina, E. (2008). Regularized estimation of large covariance matrices, The Annals of Statistics pp. 199-227. · Zbl 1132.62040 |
| [3] | Bühlmann, P., Kalisch, M. and Maathuis, M. H. (2010). Variable selection in high-dimensional linear models: partially faithful distributions and the pc-simple algorithm, Biometrika 97(2): 261-278. · Zbl 1233.62135 |
| [4] | Cai, T., Ma, Z. and Wu, Y. (2015). Optimal estimation and rank detection for sparse spiked covariance matrices, Probability theory and related fields 161(3-4): 781-815. · Zbl 1314.62130 |
| [5] | Cai, T. T., Ma, Z., Wu, Y. et al. (2013). Sparse PCA: optimal rates and adaptive estimation, The Annals of Statistics 41(6): 3074-3110. · Zbl 1288.62099 |
| [6] | Cai, T. T. and Zhou, H. H. (2010). Optimal rates of convergence for covariance matrix estimation, The Annals of Statistics 38(4): 2118-2144. · Zbl 1202.62073 |
| [7] | Cai, T. T. and Zhou, H. H. (2012). Minimax estimation of large covariance matrices under \[{l_1}\]-norm, Statistica Sinica pp. 1319-1349. · Zbl 1266.62036 |
| [8] | Chakraborty, M. and Ghosal, S. (2020). Convergence rates for Bayesian estimation and testing in monotone regression, arXiv preprint arXiv:2008.01244. · Zbl 1472.62106 |
| [9] | Chen, K., Dong, H. and Chan, K.-S. (2013). Reduced rank regression via adaptive nuclear norm penalization, Biometrika 100(4): 901-920. · Zbl 1279.62115 |
| [10] | Chen, L. and Huang, J. Z. (2012). Sparse reduced-rank regression for simultaneous dimension reduction and variable selection, Journal of the American Statistical Association 107(500): 1533-1545. · Zbl 1258.62075 |
| [11] | Demko, S., Moss, W. F. and Smith, P. W. (1984). Decay rates for inverses of band matrices, Mathematics of computation 43(168): 491-499. · Zbl 0568.15003 |
| [12] | Dunson, D. B. and Neelon, B. (2003). Bayesian inference on order-constrained parameters in generalized linear models, Biometrics 59(2): 286-295. · Zbl 1210.62097 |
| [13] | Fan, J., Rigollet, P. and Wang, W. (2015). Estimation of functionals of sparse covariance matrices, Annals of statistics 43(6): 2706. · Zbl 1327.62338 |
| [14] | Fan, J., Weng, H. and Zhou, Y. (2019). Optimal estimation of functionals of high-dimensional mean and covariance matrix, arXiv preprint arXiv:1908.07460. |
| [15] | Gelman, A., Hwang, J. and Vehtari, A. (2014). Understanding predictive information criteria for bayesian models, Statistics and computing 24(6): 997-1016. · Zbl 1332.62090 |
| [16] | Golub, G. H. and Van Loan, C. F. (2013). Matrix computations, 4th, Johns Hopkins. · Zbl 1268.65037 |
| [17] | Gunn, L. H. and Dunson, D. B. (2005). A transformation approach for incorporating monotone or unimodal constraints, Biostatistics 6(3): 434-449. · Zbl 1071.62101 |
| [18] | Johnstone, I. M. and Lu, A. Y. (2009). On consistency and sparsity for principal components analysis in high dimensions, Journal of the American Statistical Association 104(486): 682-693. · Zbl 1388.62174 |
| [19] | Kauermann, G. (1996). On a dualization of graphical gaussian models, Scandinavian journal of statistics pp. 105-116. · Zbl 0912.62006 |
| [20] | Khare, K., Rajaratnam, B. et al. (2011). Wishart distributions for decomposable covariance graph models, The Annals of Statistics 39(1): 514-555. · Zbl 1274.62369 |
| [21] | Lauritzen, S. L. (1996). Graphical models, Vol. 17, Clarendon Press. · Zbl 0907.62001 |
| [22] | Lee, K. and Lee, J. (2018). Optimal Bayesian minimax rates for unconstrained large covariance matrices, Bayesian Analysis 13(4): 1211-1229. · Zbl 1407.62043 |
| [23] | Lee, K., Lee, K. and Lee, J. (2020). Post-processed posteriors for banded covariances, arXiv preprint arXiv:2011.12627. |
| [24] | Lee, W. and Liu, Y. (2012). Simultaneous multiple response regression and inverse covariance matrix estimation via penalized gaussian maximum likelihood, Journal of multivariate analysis 111: 241-255. · Zbl 1259.62043 |
| [25] | Li, R., Liu, J. and Lou, L. (2017). Variable selection via partial correlation, Statistica Sinica 27(3): 983. · Zbl 1372.62017 |
| [26] | Lin, L. and Dunson, D. B. (2014). Bayesian monotone regression using gaussian process projection, Biometrika 101(2): 303-317. · Zbl 1452.62285 |
| [27] | Patra, S. and Dunson, D. B. (2018). Constrained Bayesian inference through posterior projections, arXiv preprint arXiv:1812.05741. |
| [28] | Press, S. J. (2012). Applied multivariate analysis: using Bayesian and frequentist methods of inference, Courier Corporation. |
| [29] | Qian, J., Tanigawa, Y., Li, R., Tibshirani, R., Rivas, M. A. and Hastie, T. (2020). Large-scale sparse regression for multiple responses with applications to UK biobank, BioRxiv. |
| [30] | Rothman, A. J., Levina, E. and Zhu, J. (2010). Sparse multivariate regression with covariance estimation, Journal of Computational and Graphical Statistics 19(4): 947-962. |
| [31] | Silva, R. and Ghahramani, Z. (2009). The hidden life of latent variables: Bayesian learning with mixed graph models, The Journal of Machine Learning Research 10: 1187-1238. · Zbl 1235.68191 |
| [32] | Transport Operation & Information Service (2021). Seoul traffic information. https://topis.seoul.go.kr/ |
| [33] | Uematsu, Y., Fan, Y., Chen, K., Lv, J. and Lin, W. (2019). SOFAR: large-scale association network learning, IEEE Transactions on Information Theory 65(8): 4924-4939. · Zbl 1432.68402 |
| [34] | Xiao, L. and Bunea, F. (2014). On the theoretic and practical merits of the banding estimator for large covariance matrices, arXiv preprint arXiv:1402.0844. |
| [35] | Yin, J. and Li, H. (2011). A sparse conditional Gaussian graphical model for analysis of genetical genomics data, The annals of applied statistics 5(4): 2630. · Zbl 1234.62151 |
| [36] | Zhao, R., Gu, X., Xue, B., Zhang, J. and Ren, W. (2018). Short period PM2.5 prediction based on multivariate linear regression model, PloS one 13(7): e0201011. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
