×

Cross: efficient low-rank tensor completion. (English) Zbl 1416.62298

Summary: The completion of tensors, or high-order arrays, attracts significant attention in recent research. Current literature on tensor completion primarily focuses on recovery from a set of uniformly randomly measured entries, and the required number of measurements to achieve recovery is not guaranteed to be optimal. In addition, the implementation of some previous methods are NP-hard. In this article, we propose a framework for low-rank tensor completion via a novel tensor measurement scheme that we name Cross. The proposed procedure is efficient and easy to implement. In particular, we show that a third-order tensor of Tucker rank-\((r_{1},r_{2},r_{3})\) in \(p_{1}\)-by-\(p_{2}\)-by-\(p_{3}\) dimensional space can be recovered from as few as \(r_{1}r_{2}r_{3}+r_{1}(p_{1}-r_{1})+r_{2}(p_{2}-r_{2})+r_{3}(p_{3}-r_{3})\) noiseless measurements, which matches the sample complexity lower bound. In the case of noisy measurements, we also develop a theoretical upper bound and the matching minimax lower bound for recovery error over certain classes of low-rank tensors for the proposed procedure. The results can be further extended to fourth or higher-order tensors. Simulation studies show that the method performs well under a variety of settings. Finally, the procedure is illustrated through a real dataset in neuroimaging.

MSC:

62H12 Estimation in multivariate analysis
62C20 Minimax procedures in statistical decision theory
62P10 Applications of statistics to biology and medical sciences; meta analysis
62H35 Image analysis in multivariate analysis
65F20 Numerical solutions to overdetermined systems, pseudoinverses
15A69 Multilinear algebra, tensor calculus
15A83 Matrix completion problems

Software:

Cross

References:

[1] Agarwal, A., Negahban, S. and Wainwright, M. J. (2012). Noisy matrix decomposition via convex relaxation: Optimal rates in high dimensions. Ann. Statist.40 1171-1197. · Zbl 1274.62219 · doi:10.1214/12-AOS1000
[2] Barak, B. and Moitra, A. (2016). Noisy tensor completion via the sum-of-squares hierarchy. In 29th Annual Conference on Learning Theory 417-445. · Zbl 1494.90065
[3] Bhojanapalli, S. and Sanghavi, S. (2015). A new sampling technique for tensors. Preprint. Available at arXiv:1502.05023. · Zbl 1371.68320
[4] Cai, T., Cai, T. T. and Zhang, A. (2016). Structured matrix completion with applications to genomic data integration. J. Amer. Statist. Assoc.111 621-633.
[5] Cai, T. T. and Zhou, W.-X. (2016). Matrix completion via max-norm constrained optimization. Electron. J. Stat.10 1493-1525. · Zbl 1342.62091 · doi:10.1214/16-EJS1147
[6] Caiafa, C. F. and Cichocki, A. (2010). Generalizing the column-row matrix decomposition to multi-way arrays. Linear Algebra Appl.433 557-573. · Zbl 1194.15025 · doi:10.1016/j.laa.2010.03.020
[7] Caiafa, C. F. and Cichocki, A. (2015). Stable, robust, and super fast reconstruction of tensors using multi-way projections. IEEE Trans. Signal Process.63 780-793. · Zbl 1394.94096 · doi:10.1109/TSP.2014.2385040
[8] Candès, E. J. and Plan, Y. (2011). Tight oracle inequalities for low-rank matrix recovery from a minimal number of noisy random measurements. IEEE Trans. Inform. Theory57 2342-2359. · Zbl 1366.90160 · doi:10.1109/TIT.2011.2111771
[9] Candès, E. J. and Tao, T. (2010). The power of convex relaxation: Near-optimal matrix completion. IEEE Trans. Inform. Theory56 2053-2080. · Zbl 1366.15021 · doi:10.1109/TIT.2010.2044061
[10] Cao, Y. and Xie, Y. (2016). Poisson matrix recovery and completion. IEEE Trans. Signal Process.64 1609-1620. · Zbl 1412.94024
[11] Cao, Y., Zhang, A. and Li, H. (2017). Multi-sample Estimation of Bacterial Composition Matrix in Metagenomics Data. Preprint. Available at arXiv:1706.02380.
[12] Gandy, S., Recht, B. and Yamada, I. (2011). Tensor completion and low-\(n\)-rank tensor recovery via convex optimization. Inverse Probl.27 025010. · Zbl 1211.15036 · doi:10.1088/0266-5611/27/2/025010
[13] Guhaniyogi, R., Qamar, S. and Dunson, D. B. (2017). Bayesian tensor regression. J. Mach. Learn. Res.18 79. · Zbl 1440.62253
[14] Hillar, C. J. and Lim, L.-H. (2013). Most tensor problems are NP-hard. J. ACM60 45. · Zbl 1281.68126 · doi:10.1145/2512329
[15] Jain, P. and Oh, S. (2014). Provable tensor factorization with missing data. In Advances in Neural Information Processing Systems1 1431-1439. MIT Press, Cambridge, MA.
[16] Jiang, X., Raskutti, G. and Willett, R. (2015). Minimax optimal rates for Poisson inverse problems with physical constraints. IEEE Trans. Inform. Theory61 4458-4474. · Zbl 1359.94106 · doi:10.1109/TIT.2015.2441072
[17] Johndrow, J. E., Bhattacharya, A. and Dunson, D. B. (2017). Tensor decompositions and sparse log-linear models. Ann. Statist.45 1-38. · Zbl 1367.62180 · doi:10.1214/15-AOS1414
[18] Karatzoglou, A., Amatriain, X., Baltrunas, L. and Oliver, N. (2010). Multiverse recommendation: N-dimensional tensor factorization for context-aware collaborative filtering. In Proceedings of the Fourth ACM Conference on Recommender Systems 79-86. ACM, New York.
[19] Keshavan, R. H., Montanari, A. and Oh, S. (2010). Matrix completion from a few entries. IEEE Trans. Inform. Theory56 2980-2998. · Zbl 1366.62111 · doi:10.1109/TIT.2010.2046205
[20] Klopp, O. (2014). Noisy low-rank matrix completion with general sampling distribution. Bernoulli20 282-303. · Zbl 1400.62115 · doi:10.3150/12-BEJ486
[21] Kolda, T. G. and Bader, B. W. (2009). Tensor decompositions and applications. SIAM Rev.51 455-500. · Zbl 1173.65029 · doi:10.1137/07070111X
[22] Koltchinskii, V., Lounici, K. and Tsybakov, A. B. (2011). Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion. Ann. Statist.39 2302-2329. · Zbl 1231.62097 · doi:10.1214/11-AOS894
[23] Kressner, D., Steinlechner, M. and Vandereycken, B. (2014). Low-rank tensor completion by Riemannian optimization. BIT54 447-468. · Zbl 1300.65040 · doi:10.1007/s10543-013-0455-z
[24] Krishnamurthy, A. and Singh, A. (2013). Low-rank matrix and tensor completion via adaptive sampling. In Advances in Neural Information Processing Systems 836-844.
[25] Li, N. and Li, B. (2010). Tensor completion for on-board compression of hyperspectral images. In 2010 IEEE International Conference on Image Processing 517-520. IEEE, New York.
[26] Li, L. and Zhang, X. (2017). Parsimonious tensor response regression. J. Amer. Statist. Assoc.112 1131-1146.
[27] Li, X., Zhou, H. and Li, L. (2013). Tucker tensor regression and neuroimaging analysis. Preprint. Available at arXiv:1304.5637.
[28] Li, L., Chen, Z., Wang, G., Chu, J. and Gao, H. (2014). A tensor PRISM algorithm for multi-energy CT reconstruction and comparative studies. J. X-Ray Sci. Technol.22 147-163.
[29] Liu, J., Musialski, P., Wonka, P. and Ye, J. (2013). Tensor completion for estimating missing values in visual data. IEEE Trans. Pattern Anal. Mach. Intell.35 208-220.
[30] Mahoney, M. W., Maggioni, M. and Drineas, P. (2008). Tensor-CUR decompositions for tensor-based data. SIAM J. Matrix Anal. Appl.30 957-987. · Zbl 1168.65340 · doi:10.1137/060665336
[31] Mu, C., Huang, B., Wright, J. and Goldfarb, D. (2014). Square deal: Lower bounds and improved relaxations for tensor recovery. In ICML 73-81.
[32] Negahban, S. and Wainwright, M. J. (2011). Estimation of (near) low-rank matrices with noise and high-dimensional scaling. Ann. Statist.39 1069-1097. · Zbl 1216.62090 · doi:10.1214/10-AOS850
[33] Negahban, S. and Wainwright, M. J. (2012). Restricted strong convexity and weighted matrix completion: Optimal bounds with noise. J. Mach. Learn. Res.13 1665-1697. · Zbl 1436.62204
[34] Nowak, R. D. and Kolaczyk, E. D. (2000). A statistical multiscale framework for Poisson inverse problems. IEEE Trans. Inform. Theory46 1811-1825. · Zbl 0999.94004 · doi:10.1109/18.857793
[35] Oseledets, I. V., Savostianov, D. V. and Tyrtyshnikov, E. E. (2008). Tucker dimensionality reduction of three-dimensional arrays in linear time. SIAM J. Matrix Anal. Appl.30 939-956. · Zbl 1180.15025 · doi:10.1137/060655894
[36] Oseledets, I. V. and Tyrtyshnikov, E. E. (2009). Breaking the curse of dimensionality, or how to use SVD in many dimensions. SIAM J. Sci. Comput.31 3744-3759. · Zbl 1200.65028 · doi:10.1137/090748330
[37] Pimentel-Alarcón, D. L., Boston, N. and Nowak, R. D. (2016). A characterization of deterministic sampling patterns for low-rank matrix completion. IEEE J. Sel. Top. Signal Process.10 623-636.
[38] Raskutti, G., Yuan, M. and Chen, H. (2017). Convex regularization for high-dimensional multi-response tensor regression. Preprint. Available at arXiv:1512.01215v2.
[39] Rauhut, H., Schneider, R. and Stojanac, Ž. (2017). Low rank tensor recovery via iterative hard thresholding. Linear Algebra Appl.523 220-262. · Zbl 1372.65130 · doi:10.1016/j.laa.2017.02.028
[40] Recht, B. (2011). A simpler approach to matrix completion. J. Mach. Learn. Res.12 3413-3430. · Zbl 1280.68141
[41] Rendle, S. and Schmidt-Thieme, L. (2010). Pairwise interaction tensor factorization for personalized tag recommendation. In Proceedings of the Third ACM International Conference on Web Search and Data Mining 81-90. ACM, New York.
[42] Richard, E. and Montanari, A. (2014). A statistical model for tensor PCA. In Advances in Neural Information Processing Systems 2897-2905.
[43] Rohde, A. and Tsybakov, A. B. (2011). Estimation of high-dimensional low-rank matrices. Ann. Statist.39 887-930. · Zbl 1215.62056 · doi:10.1214/10-AOS860
[44] Rudelson, M. and Vershynin, R. (2007). Sampling from large matrices: An approach through geometric functional analysis. J. ACM54 21. · Zbl 1326.68333 · doi:10.1145/1255443.1255449
[45] Semerci, O., Hao, N., Kilmer, M. E. and Miller, E. L. (2014). Tensor-based formulation and nuclear norm regularization for multienergy computed tomography. IEEE Trans. Image Process.23 1678-1693. · Zbl 1374.94335 · doi:10.1109/TIP.2014.2305840
[46] Shah, P., Rao, N. and Tang, G. (2015). Optimal low-rank tensor recovery from separable measurements: Four contractions suffice. Preprint. Available at arXiv:1505.04085.
[47] Srebro, N. and Shraibman, A. (2005). Rank, trace-norm and max-norm. In Learning Theory. Lecture Notes in Computer Science3559 545-560. Springer, Berlin. · Zbl 1137.68563
[48] Sun, W. W. and Li, L. (2016). Sparse low-rank tensor response regression. Preprint. Available at arXiv:1609.04523.
[49] Sun, W. W., Lu, J., Liu, H. and Cheng, G. (2017). Provable sparse tensor decomposition. J. R. Stat. Soc. Ser. B. Stat. Methodol.79 899-916. · Zbl 1411.62158
[50] Tucker, L. R. (1966). Some mathematical notes on three-mode factor analysis. Psychometrika31 279-311.
[51] Wagner, A. and Zuk, O. (2015). Low-rank matrix recovery from row-and-column affine measurements. In Proceedings of the 32nd International Conference on Machine Learning (ICML-15) 2012-2020.
[52] Wang, Y. and Singh, A. (2015). Provably correct algorithms for matrix column subset selection with selectively sampled data. Preprint. Available at arXiv:1505.04343. · Zbl 1467.68154
[53] Yuan, M. and Zhang, C.-H. (2016). On tensor completion via nuclear norm minimization. Found. Comput. Math.16 1031-1068. · Zbl 1378.90066 · doi:10.1007/s10208-015-9269-5
[54] Yuan, M. and Zhang, C.-H. (2017). Incoherent tensor norms and their applications in higher order tensor completion. IEEE Trans. Inform. Theory63 6753-6766. · Zbl 1391.94469 · doi:10.1109/TIT.2017.2724549
[55] Zhang, A. (2019). Supplement to “Cross: Efficient low-rank tensor completion.” DOI:10.1214/18-AOS1694SUPP. · Zbl 1416.62298 · doi:10.1214/18-AOS1694
[56] Zhang, A. and Xia, D. (2017). Tensor SVD: Statistical and computational limits. IEEE Trans. Inform. Theory64 7311-7338. · Zbl 1432.62176 · doi:10.1109/TIT.2018.2841377
[57] Zhou, H., Li, L. and Zhu, H. (2013). Tensor regression with applications in neuroimaging data analysis. J. Amer. Statist. Assoc.108 540-552. · Zbl 06195959 · doi:10.1080/01621459.2013.776499
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.