Singular vector and singular subspace distribution for the matrix denoising model. (English) Zbl 1459.60011
Summary: In this paper, we study the matrix denoising model \(Y = S + X\), where \(S\) is a low rank deterministic signal matrix and \(X\) is a random noise matrix, and both are \(M \times n\). In the scenario that \(M\) and \(n\) are comparably large and the signals are supercritical, we study the fluctuation of the outlier singular vectors of \(Y\), under fully general assumptions on the structure of \(S\) and the distribution of \(X\). More specifically, we derive the limiting distribution of angles between the principal singular vectors of \(Y\) and their deterministic counterparts, the singular vectors of \(S\). Further, we also derive the distribution of the distance between the subspace spanned by the principal singular vectors of \(Y\) and that spanned by the singular vectors of \(S\). It turns out that the limiting distributions depend on the structure of the singular vectors of \(S\) and the distribution of \(X\), and thus they are nonuniversal. Statistical applications of our results to singular vector and singular subspace inferences are also discussed.
MSC:
| 60B20 | Random matrices (probabilistic aspects) |
| 15B52 | Random matrices (algebraic aspects) |
| 62G10 | Nonparametric hypothesis testing |
| 62H10 | Multivariate distribution of statistics |
| 62H25 | Factor analysis and principal components; correspondence analysis |
Keywords:
random matrix; singular vector; singular subspace; matrix denoising model; signal-plus-noise model; nonuniversalityReferences:
| [1] | Anderson, T. W. (1963). Asymptotic theory for principal component analysis. Ann. Math. Stat. 34 122-148. · Zbl 0202.49504 · doi:10.1214/aoms/1177704248 |
| [2] | Bai, Z. and Silverstein, J. W. (2010). Spectral Analysis of Large Dimensional Random Matrices, 2nd ed. Springer Series in Statistics. Springer, New York. · Zbl 1301.60002 |
| [3] | Bai, Z. and Yao, J. (2008). Central limit theorems for eigenvalues in a spiked population model. Ann. Inst. Henri Poincaré Probab. Stat. 44 447-474. · Zbl 1274.62129 · doi:10.1214/07-AIHP118 |
| [4] | Bai, Z. and Yao, J. (2012). On sample eigenvalues in a generalized spiked population model. J. Multivariate Anal. 106 167-177. · Zbl 1301.62049 |
| [5] | Baik, J., Ben Arous, G. and Péché, S. (2005). Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices. Ann. Probab. 33 1643-1697. · Zbl 1086.15022 · doi:10.1214/009117905000000233 |
| [6] | Baik, J. and Silverstein, J. W. (2006). Eigenvalues of large sample covariance matrices of spiked population models. J. Multivariate Anal. 97 1382-1408. · Zbl 1220.15011 · doi:10.1016/j.jmva.2005.08.003 |
| [7] | Bao, Z., Ding, X., Wang, J. and Wang, K. (2019). Principal components of spiked covariance matrices in the supercritical regime. Available at arXiv:1907.12251. |
| [8] | Bao, Z., Ding, X. and Wang, K. (2021). Supplement to “Singular vector and singular subspace distribution for the matrix denoising model.” https://doi.org/10.1214/20-AOS1960SUPP |
| [9] | Bao, Z., Hu, J., Pan, G. and Zhou, W. (2019). Canonical correlation coefficients of high-dimensional Gaussian vectors: Finite rank case. Ann. Statist. 47 612-640. · Zbl 1416.62313 · doi:10.1214/18-AOS1704 |
| [10] | Bao, Z., Pan, G. and Zhou, W. (2015). Universality for the largest eigenvalue of sample covariance matrices with general population. Ann. Statist. 43 382-421. · Zbl 1408.60006 · doi:10.1214/14-AOS1281 |
| [11] | Benaych-Georges, F., Guionnet, A. and Maida, M. (2011). Fluctuations of the extreme eigenvalues of finite rank deformations of random matrices. Electron. J. Probab. 16 1621-1662. · Zbl 1245.60007 · doi:10.1214/EJP.v16-929 |
| [12] | Benaych-Georges, F. and Nadakuditi, R. R. (2011). The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices. Adv. Math. 227 494-521. · Zbl 1226.15023 · doi:10.1016/j.aim.2011.02.007 |
| [13] | Benaych-Georges, F. and Nadakuditi, R. R. (2012). The singular values and vectors of low rank perturbations of large rectangular random matrices. J. Multivariate Anal. 111 120-135. · Zbl 1252.15039 · doi:10.1016/j.jmva.2012.04.019 |
| [14] | Bloemendal, A., Erdos, L., Knowles, A., Yau, H.-T. and Yin, J. (2014). Isotropic local laws for sample covariance and generalized Wigner matrices. Electron. J. Probab. 19 no. 33, 53. · Zbl 1288.15044 · doi:10.1214/EJP.v19-3054 |
| [15] | Bloemendal, A., Knowles, A., Yau, H.-T. and Yin, J. (2016). On the principal components of sample covariance matrices. Probab. Theory Related Fields 164 459-552. · Zbl 1339.15023 · doi:10.1007/s00440-015-0616-x |
| [16] | Bloemendal, A. and Virág, B. (2013). Limits of spiked random matrices I. Probab. Theory Related Fields 156 795-825. · Zbl 1356.60014 · doi:10.1007/s00440-012-0443-2 |
| [17] | Bloemendal, A. and Virág, B. (2016). Limits of spiked random matrices II. Ann. Probab. 44 2726-2769. · Zbl 1396.60004 · doi:10.1214/15-AOP1033 |
| [18] | Cai, T. T. and Zhang, A. (2018). Rate-optimal perturbation bounds for singular subspaces with applications to high-dimensional statistics. Ann. Statist. 46 60-89. · Zbl 1395.62122 · doi:10.1214/17-AOS1541 |
| [19] | Cape, J., Tang, M. and Priebe, C. E. (2019). Signal-plus-noise matrix models: Eigenvector deviations and fluctuations. Biometrika 106 243-250. · Zbl 1506.62300 · doi:10.1093/biomet/asy070 |
| [20] | Cape, J., Tang, M. and Priebe, C. E. (2019). The two-to-infinity norm and singular subspace geometry with applications to high-dimensional statistics. Ann. Statist. 47 2405-2439. · Zbl 1470.62065 · doi:10.1214/18-AOS1752 |
| [21] | Capitaine, M. (2018). Limiting eigenvectors of outliers for spiked information-plus-noise type matrices. In Séminaire de Probabilités XLIX. Lecture Notes in Math. 2215 119-164. Springer, Cham. · Zbl 1451.15022 |
| [22] | Capitaine, M. and Donati-Martin, C. (2017). Spectrum of deformed random matrices and free probability. In Advanced Topics in Random Matrices. Panor. Synthèses 53 151-190. Soc. Math. France, Paris. · Zbl 1417.60007 |
| [23] | Capitaine, M. and Donati-Martin, C. (2018). Non universality of fluctuations of outlier eigenvectors for block diagonal deformations of Wigner matrices. Available at arXiv:1807.07773. |
| [24] | Capitaine, M., Donati-Martin, C. and Féral, D. (2009). The largest eigenvalues of finite rank deformation of large Wigner matrices: Convergence and nonuniversality of the fluctuations. Ann. Probab. 37 1-47. · Zbl 1163.15026 · doi:10.1214/08-AOP394 |
| [25] | Capitaine, M., Donati-Martin, C. and Féral, D. (2012). Central limit theorems for eigenvalues of deformations of Wigner matrices. Ann. Inst. Henri Poincaré Probab. Stat. 48 107-133. · Zbl 1237.60007 · doi:10.1214/10-AIHP410 |
| [26] | Ding, X. (2020). High dimensional deformed rectangular matrices with applications in matrix denoising. Bernoulli 26 387-417. · Zbl 1439.15013 · doi:10.3150/19-BEJ1129 |
| [27] | Ding, X. and Yang, F. (2019). Spiked separable covariance matrices and principal components. Available at arXiv:1905.13060. |
| [28] | Donoho, D. and Gavish, M. (2014). Minimax risk of matrix denoising by singular value thresholding. Ann. Statist. 42 2413-2440. · Zbl 1310.62014 · doi:10.1214/14-AOS1257 |
| [29] | El Karoui, N. (2007). Tracy-Widom limit for the largest eigenvalue of a large class of complex sample covariance matrices. Ann. Probab. 35 663-714. · Zbl 1117.60020 · doi:10.1214/009117906000000917 |
| [30] | Erdos, L., Knowles, A. and Yau, H.-T. (2013). Averaging fluctuations in resolvents of random band matrices. Ann. Henri Poincaré 14 1837-1926. · Zbl 1296.15020 · doi:10.1007/s00023-013-0235-y |
| [31] | Fan, J., Sun, Q., Zhou, W.-X. and Zhu, Z. (2018). Principal component analysis for big data. Available at arXiv:1801.01602. |
| [32] | Fan, J., Wang, W. and Zhong, Y. (2017). An \(\ell_{\infty}\) eigenvector perturbation bound and its application to robust covariance estimation. J. Mach. Learn. Res. 18 Paper No. 207, 42. · Zbl 1473.15015 |
| [33] | Fan, J. and Zhong, Y. (2018). Optimal subspace estimation using overidentifying vectors via generalized method of moments. Available at arXiv:1805.02826. |
| [34] | Fan, Z., Johnstone, I. and Sun, Y. (2018). Spiked covariances and principal components analysis in high-dimensional random effects models. Available at arXiv:1806.09529. |
| [35] | Féral, D. and Péché, S. (2007). The largest eigenvalue of rank one deformation of large Wigner matrices. Comm. Math. Phys. 272 185-228. · Zbl 1136.82016 |
| [36] | Golub, G. H. and Van Loan, C. F. (2012). Matrix Computations, 3rd ed. Johns Hopkins Studies in the Mathematical Sciences. Johns Hopkins Univ. Press, Baltimore, MD. |
| [37] | Hachem, W., Loubaton, P., Mestre, X., Najim, J. and Vallet, P. (2013). A subspace estimator for fixed rank perturbations of large random matrices. J. Multivariate Anal. 114 427-447. · Zbl 1359.62185 · doi:10.1016/j.jmva.2012.08.006 |
| [38] | Heckel, R., Tschannen, M. and Bölcskei, H. (2017). Dimensionality-reduced subspace clustering. Inf. Inference 6 246-283. · Zbl 1383.62177 · doi:10.1093/imaiai/iaw021 |
| [39] | Huang, J., Qiu, Q. and Calderbank, R. (2016). The role of principal angles in subspace classification. IEEE Trans. Signal Process. 64 1933-1945. · Zbl 1414.94250 · doi:10.1109/TSP.2015.2500889 |
| [40] | Kay, S. M. (1998). Fundamentals of Statistical Signal Processing, Volume 2: Detection Theory. Prentice-Hall, New York. |
| [41] | Knowles, A. and Yin, J. (2013). The isotropic semicircle law and deformation of Wigner matrices. Comm. Pure Appl. Math. 66 1663-1750. · Zbl 1290.60004 · doi:10.1002/cpa.21450 |
| [42] | Knowles, A. and Yin, J. (2014). The outliers of a deformed Wigner matrix. Ann. Probab. 42 1980-2031. · Zbl 1306.15034 · doi:10.1214/13-AOP855 |
| [43] | Knowles, A. and Yin, J. (2017). Anisotropic local laws for random matrices. Probab. Theory Related Fields 169 257-352. · Zbl 1382.15051 · doi:10.1007/s00440-016-0730-4 |
| [44] | Koltchinskii, V., Löffler, M. and Nickl, R. (2020). Efficient estimation of linear functionals of principal components. Ann. Statist. 48 464-490. · Zbl 1440.62232 · doi:10.1214/19-AOS1816 |
| [45] | Koltchinskii, V. and Lounici, K. (2016). Asymptotics and concentration bounds for bilinear forms of spectral projectors of sample covariance. Ann. Inst. Henri Poincaré Probab. Stat. 52 1976-2013. · Zbl 1353.62053 · doi:10.1214/15-AIHP705 |
| [46] | Koltchinskii, V. and Lounici, K. (2017). Normal approximation and concentration of spectral projectors of sample covariance. Ann. Statist. 45 121-157. · Zbl 1367.62175 · doi:10.1214/16-AOS1437 |
| [47] | Lee, J. O. and Schnelli, K. (2018). Local law and Tracy-Widom limit for sparse random matrices. Probab. Theory Related Fields 171 543-616. · Zbl 1429.60012 · doi:10.1007/s00440-017-0787-8 |
| [48] | Lee, M., Shen, H., Huang, J. Z. and Marron, J. S. (2010). Biclustering via sparse singular value decomposition. Biometrics 66 1087-1095. · Zbl 1233.62182 · doi:10.1111/j.1541-0420.2010.01392.x |
| [49] | Levin, A. and Nadler, B. (2011). Natural image denoising: Optimality and inherent bounds. In 2011 IEEE Conference On Computer Vision And Pattern Recognition (CVPR) 40. |
| [50] | Marcenko, V. A. and Pastur, L. A. (1967). The spectrum of random matrices. Teor. Funkciĭ Funkcional. Anal. i Priložen. Vyp. 4 122-145. |
| [51] | Nadler, B. and Johnstone, I. M. (2011). On the distribution of Roy’s Largest Root Test in MANOVA and in signal detection in noise. Technical report, Department of Statistics, Stanford University. |
| [52] | O’Rourke, S., Vu, V. and Wang, K. (2018). Random perturbation of low rank matrices: Improving classical bounds. Linear Algebra Appl. 540 26-59. · Zbl 1380.65076 · doi:10.1016/j.laa.2017.11.014 |
| [53] | Paul, D. (2007). Asymptotics of sample eigenstructure for a large dimensional spiked covariance model. Statist. Sinica 17 1617-1642. · Zbl 1134.62029 |
| [54] | Péché, S. (2006). The largest eigenvalue of small rank perturbations of Hermitian random matrices. Probab. Theory Related Fields 134 127-173. · Zbl 1088.15025 · doi:10.1007/s00440-005-0466-z |
| [55] | Peterfreund, E. and Gavish, M. (2018). Multidimensional scaling of noisy high dimensional data. Available at arXiv:1801.10229. |
| [56] | Pillai, N. S. and Yin, J. (2014). Universality of covariance matrices. Ann. Appl. Probab. 24 935-1001. · Zbl 1296.15021 · doi:10.1214/13-AAP939 |
| [57] | Pizzo, A., Renfrew, D. and Soshnikov, A. (2013). On finite rank deformations of Wigner matrices. Ann. Inst. Henri Poincaré Probab. Stat. 49 64-94. · Zbl 1278.60014 · doi:10.1214/11-AIHP459 |
| [58] | Wang, W., Miguel, A. C.-P. and Lu, Z. (2011). A denoising view of matrix completion. Adv. Neural Inf. Process. Syst. 24 334-342. |
| [59] | Xia, D. (2019). Confidence region of singular subspaces for low-rank matrix regression. IEEE Trans. Inf. Theory 65 7437-7459. · Zbl 1433.94034 · doi:10.1109/TIT.2019.2924900 |
| [60] | Yang, D., Ma, Z. and Buja, A. (2014). A sparse singular value decomposition method for high-dimensional data. J. Comput. Graph. Statist. 23 923-942. |
| [61] | Yang, D., Ma, Z. and Buja, A. (2016). Rate optimal denoising of simultaneously sparse and low rank matrices. J. Mach. Learn. Res. 17 Paper No. 92, 27. · Zbl 1368.62144 |
| [62] | Zahernia, A., Dehghani, M. J. and Javidan, R. (2011). MUSIC algorithm for DOA estimation using MIMO arrays. In 2011 6th International Conference on Telecommunication Systems, Services, and Applications (TSSA) 149-153. |
| [63] | Zhong, Y. · Zbl 1396.90068 · doi:10.1137/17M1122025 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
