Estimation of sparse covariance matrix via non-convex regularization. (English) Zbl 07846368
Summary: Estimation of high-dimensional sparse covariance matrix is one of the fundamental and important problems in multivariate analysis and has a wide range of applications in many fields. This paper presents a novel method for sparse covariance matrix estimation via solving a non-convex regularization optimization problem. We establish the asymptotic properties of the proposed estimator and develop a multi-stage convex relaxation method to find an effective estimator. The multi-stage convex relaxation method guarantees any accumulation point of the sequence generated is a first-order stationary point of the non-convex optimization. Moreover, the error bounds of the first two stage estimators of the multi-stage convex relaxation method are derived under some regularity conditions. The numerical results show that our estimator outperforms the state-of-the-art estimators and has a high degree of sparsity on the premise of its effectiveness.
MSC:
| 62Hxx | Multivariate analysis |
| 62H12 | Estimation in multivariate analysis |
| 62F12 | Asymptotic properties of parametric estimators |
References:
| [1] | Avella-Medina, M.; Battey, H.; Fan, J.; Li, Q., Robust estimation of high-dimensional covariance and precision matrices, Biometrika, 105, 271-284, 2018 · Zbl 07072412 |
| [2] | Bickel, P. J.; Levina, E., Covariance regularization by thresholding, Ann. Statist., 36, 2577-2604, 2008 · Zbl 1196.62062 |
| [3] | Cai, T. T.; Liu, W., Adaptive thresholding for sparse covariance matrix estimation, J. Amer. Statist. Assoc., 106, 672-684, 2011 · Zbl 1232.62086 |
| [4] | Cai, T. T.; Zhang, A., Minimax rate-optimal estimation of high-dimensional covariance matrices with incomplete data, J. Multivariate Anal., 150, 55-74, 2016 · Zbl 1347.62088 |
| [5] | Cai, T. T.; Zhou, H. H., Minimax estimation of large covariance matrices under \(\ell_1\) norm, Statist. Sinica, 22, 1319-1378, 2012 · Zbl 1266.62036 |
| [6] | Cho, S.; Katayama, S.; Choi, Y.-G.; Lim, J., Positive-defnite modifcation of a covariance matrix by minimizing the matrix \(l_\infty\) norm with applications to portfolio optimization, AStA Adv. Stat. Anal., 105, 601-627, 2021 · Zbl 1478.62118 |
| [7] | Choi, Y.-G.; Lim, J.; Roy, A.; Park, J., Fixed support positive-definite modification of covariance matrix estimators via linear shrinkage, J. Multivariate Anal., 171, 234-249, 2019 · Zbl 1417.62141 |
| [8] | Cui, Y.; Leng, C.; Sun, D., Sparse estimation of high-dimensional correlation matrices, Comput. Statist. Data Anal., 93, 390-403, 2016 · Zbl 1468.62044 |
| [9] | El Karoui, N., Operator norm consistent estimation of large dimensional sparse covariance matrices, Ann. Statist., 36, 2717-2756, 2008 · Zbl 1196.62064 |
| [10] | Fan, J.; Fan, Y.; Barut, E., Adaptive robust variable selection, Ann. Statist., 42, 1, 324-351, 2014 · Zbl 1296.62144 |
| [11] | Fan, J.; Li, R., Variable selection via nonconcave penalized likelihood and its oracle properties, J. Amer. Statist. Assoc., 96, 456, 1348-1360, 2001 · Zbl 1073.62547 |
| [12] | Fan, J.; Li, Q.; Wang, Y., Estimation of high dimensional mean regression in the absence of symmetry and light tail assumptions, J. R. Stat. Soc. Ser. B Stat. Methodol., 79, 247-265, 2017 · Zbl 1414.62178 |
| [13] | Fan, J.; Liao, Y.; Mincheva, M., Large covariance estimation by thresholding principal orthogonal complements, J. R. Stat. Soc. Ser. B Stat. Methodol., 75, 603-680, 2013 · Zbl 1411.62138 |
| [14] | He, Y.; Liu, P.; Zhang, X.; Zhou, W., Robust covariance estimation for high-dimensional compositional data with application to microbial communities analysis, Stat. Med., 40, 15, 3499-3515, 2021 |
| [15] | Kim, R.; Pourahmadi, M.; Garcia, T. P., Positive-definite thresholding estimators of covariance matrices with zeros, J. Multivariate Anal., 197, Article 105186 pp., 2023 · Zbl 07723936 |
| [16] | Li, D.; Srinivasan, A.; Chen, Q.; Xue, L., Robust covariance matrix estimation for high-dimensional compositional data with application to sales data analysis, J. Bus. Econom. Statist., 2022 |
| [17] | Mordukhovich, B. S.; Nam, N. M., An Easy Path to Convex Analysis and Applications, 2014, Morgan and Claypool, New York · Zbl 1284.49002 |
| [18] | Pan, L.; Chen, X., Group sparse optimization for images recovery using capped folded concave functions, SIAM J. Imaging Sci., 14, 1, 1-25, 2021 · Zbl 1474.90507 |
| [19] | Rockafellar, R.; Wets, R. J.-B., Variational Analysis, 1998, Springer: Springer New York · Zbl 0888.49001 |
| [20] | Rothman, A. J., Positive definite estimators of large covariance matrices, Biometrika, 99, 733-740, 2012 · Zbl 1437.62595 |
| [21] | Rothman, A. J.; Levina, E.; Zhu, J., Generalized thresholding of large covariance matrices, J. Amer. Statist. Assoc., 104, 177-186, 2009 · Zbl 1388.62170 |
| [22] | Wen, F.; Chu, L.; Ying, R.; Liu, P., Fast and positive definite estimation of large covariance matrix for high-dimensional data analysis, IEEE Trans. Big Data, 7, 3, 603-609, 2021 |
| [23] | Wen, F.; Yang, Y.; Liu, P.; Qiu, R. C., Positive definite estimation of large covariance matrix using generalized nonconvex penalities, IEEE Access, 4, 4168-4182, 2016 |
| [24] | Xue, L.; Ma, S.; Zou, H., Positive-definite \(L_1\) penalized estimation of large covariance matrices, J. Amer. Statist. Assoc., 107, 1480-1491, 2012 · Zbl 1258.62063 |
| [25] | Zhang, T., Analysis of multi-stage convex relaxation for sparse regularization, J. Mach. Learn. Res., 11, 35, 1081-1107, 2010 · Zbl 1242.68262 |
| [26] | Zhang, C., Nearly unbiased variable selection under minimax concave penalty, Ann. Statist., 38, 2, 894-942, 2010 · Zbl 1183.62120 |
| [27] | Zhang, T., Multi-stage convex relaxation for feature selection, Bernoulli, 19, 5B, 2277-2293, 2013 · Zbl 1359.62293 |
| [28] | Zou, H.; Li, R., One-step sparse estimates in nonconcave penalized likelihood models, Ann. Statist., 36, 4, 1509-1533, 2008 · Zbl 1142.62027 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
