×

Trading off \(1\)-norm and sparsity against rank for linear models using mathematical optimization: \(1\)-norm minimizing partially reflexive ah-symmetric generalized inverses. (English) Zbl 1497.90194

Summary: The M-P (Moore-Penrose) pseudoinverse has as a key application the computation of least-squares solutions of inconsistent systems of linear equations. Irrespective of whether a given input matrix is sparse, its M-P pseudoinverse can be dense, potentially leading to high computational burden, especially when we are dealing with high-dimensional matrices. The M-P pseudoinverse is uniquely characterized by four properties, but only two of them need to be satisfied for the computation of least-squares solutions. M. Fampa and J. Lee [Oper. Res. Lett. 46, No. 6, 605–610 (2018; Zbl 1476.15004)] and L. Xu et al. [SIAM J. Optim. 31, No. 3, 1722–1747 (2021; Zbl 1472.90102)] propose local-search procedures to construct sparse block-structured generalized inverses that satisfy the two key M-P properties, plus one more (the so-called reflexive property). That additional M-P property is equivalent to imposing a minimum-rank condition on the generalized inverse. (Vector) \(1\)-norm minimization is used to induce sparsity and, importantly, to keep the magnitudes of entries under control for the generalized-inverses constructed. Here, we investigate the trade-off between low \(1\)-norm and low rank for generalized inverses that can be used in the computation of least-squares solutions. We propose several algorithmic approaches that start from a \(1\)-norm minimizing generalized inverse that satisfies the two key M-P properties, and gradually decrease its rank, by iteratively imposing the reflexive property. The algorithms iterate until the generalized inverse has the least possible rank. During the iterations, we produce intermediate solutions, trading off low \(1\)-norm (and typically high sparsity) against low rank.

MSC:

90C30 Nonlinear programming
15A09 Theory of matrix inversion and generalized inverses

References:

[1] Chandrasekaran, Venkat; Sanghave, Sujay; Parrilo, Pablo A.; Willsky, Alan S., Rank-sparsity incoherence for matrix decomposition, SIAM J. Optim., 21, 2, 572-596 (2011) · Zbl 1226.90067 · doi:10.1137/090761793
[2] Dokmanić, Ivan; Gribonval, Rémi, Beyond Moore-Penrose part I: generalized inverses that minimize matrix norms (2017) · Zbl 1407.15005
[3] Fampa, Marcia; Lee, Jon, On sparse reflexive generalized inverses, Oper. Res. Lett., 46, 6, 605-610 (2018) · Zbl 1476.15004 · doi:10.1016/j.orl.2018.09.005
[4] Fampa, Marcia; Lee, Jon; Ponte, Gabriel; Xu, Luze, Experimental analysis of local search for sparse reflexive generalized inverses (2019) · Zbl 1489.15011
[5] Fuentes, Victor K.; Fampa, Marcia; Lee, Jon, Sparse pseudoinverses via LP and SDP relaxations of Moore-Penrose, CLAIO 2016, 343-350 (2016)
[6] Golub, Gene H.; Van Loan, Charles F., Matrix Computations (3rd Ed.) (1996), Johns Hopkins University Press · Zbl 0733.65016
[7] Mohan, Karthik; Fazel, Maryam, Iterative reweighted algorithms for matrix rank minimization, The Journal of Machine Learning Research, 13, 3441-3473 (2012) · Zbl 1436.65055
[8] Penrose, Roger, A generalized inverse for matrices, Proc. Camb. Philos. Soc., 51, 406-413 (1955) · Zbl 0065.24603 · doi:10.1017/S0305004100030401
[9] Recht, Benjamin; Fazel, Maryam; Parrilo, Pablo A., Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization, SIAM Rev., 52, 3, 471-501 (2010) · Zbl 1198.90321 · doi:10.1137/070697835
[10] Rohde, Charles A., Contributions to the theory, computation and application of generalized inverses (1964)
[11] Xu, Luze; Fampa, Marcia; Lee, Jon; Ponte, Gabriel, Approximate 1-norm minimization and minimum-rank structured sparsity for various generalized inverses via local search (2021) · Zbl 1472.90102
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.