A geometric proximal gradient method for sparse least squares regression with probabilistic simplex constraint. (English) Zbl 1492.65165
Summary: In this paper, we consider the sparse least squares regression problem with probabilistic simplex constraint. Due to the probabilistic simplex constraint, one could not apply directly the \(\ell_1\)-regularization to the considered regression model. To find a sparse solution, we reformulate the sparse least squares regression problem as a nonconvex and nonsmooth \(\ell_1\)-regularized minimization problem over the unit sphere. Then we propose a geometric proximal gradient method for solving the regularized problem with a varied regularized parameter, where the explicit expression of the global solution to every involved subproblem is obtained. The global convergence of the proposed method is established under some mild assumptions. Some numerical results are reported to illustrate the effectiveness of the proposed algorithm.
MSC:
| 65K05 | Numerical mathematical programming methods |
| 90C25 | Convex programming |
| 90C26 | Nonconvex programming, global optimization |
Keywords:
sparse least squares regression; probabilistic simplex constraint; \(\ell_1\)-regularization; geometric proximal gradient methodReferences:
| [1] | Absil, P-A; Mahony, R.; Sepulchre, R., Optimization Algorithms on Matrix Manifolds (2008), Princeton, NJ: Princeton University Press, Princeton, NJ · Zbl 1147.65043 · doi:10.1515/9781400830244 |
| [2] | Absil, P-A; Malick, J., Projection-like retractions on matrix manifolds, SIAM J. Optim., 22, 135-158 (2012) · Zbl 1248.49055 · doi:10.1137/100802529 |
| [3] | Attouch, H.; Bolte, J.; Redont, P.; Soubeyran, A., Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka-Łojasiewicz inequality, Math. Oper. Res., 35, 438-457 (2010) · Zbl 1214.65036 · doi:10.1287/moor.1100.0449 |
| [4] | Beck, A., First-Order Methods in Optimization (2017), Philadelphia: SIAM, Philadelphia · Zbl 1384.65033 · doi:10.1137/1.9781611974997 |
| [5] | Bertsekas, DP, Nonlinear Programming (1999), Belmont Massachusetts: Athena Scientific, Belmont Massachusetts · Zbl 1015.90077 |
| [6] | Bioucas-Dias, J. M., Figueiredo, M. A. T.: Alternating direction algorithms for constrained sparse regression: Application to hyperspectral unmixing. In: 2010 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing, pp. 1-4 (2010) |
| [7] | Bolte, J.; Sabach, S.; Teboulle, M., Proximal alternating linearized minimization for nonconvex and nonsmooth problems, Math. Program., 146, 459-494 (2014) · Zbl 1297.90125 · doi:10.1007/s10107-013-0701-9 |
| [8] | Boyd, S.; Vandenberghe, L., Convex Optimization (2004), Cambridge, UK: Cambridge University Press, Cambridge, UK · Zbl 1058.90049 · doi:10.1017/CBO9780511804441 |
| [9] | Chen, X.; Ching, WK; Chen, XS; Cong, Y.; Tsing, NK, Construction of probabilistic Boolean networks from a prescribed transition probability matrix: A maximum entropy rate approach, East Asian J. Appl. Math., 1, 132-154 (2011) · Zbl 1287.65045 · doi:10.4208/eajam.080310.200910a |
| [10] | Chen, X.; Jiang, H.; Ching, WK, On construction of sparse probabilistic Boolean networks, East Asian J. Appl. Math., 2, 1-18 (2012) · Zbl 1284.60141 · doi:10.4208/eajam.030511.060911a |
| [11] | Ching, W. K., Chen, X., Tsing, N. K., Leung, H. Y.: A heuristic method for generating probabilistic Boolean networks from a prescribed transition probability matrix. In: Proc. 2nd Symposium on Optimization and Systems Biology (OSB’08), Lijiang, China, October 31-November 3, pp. 271-278 (2008) |
| [12] | Ching, WK; Cong, Y., A new optimization model for the construction of Markov chains, International Joint Conference on Computational Sciences and Optimization, 2009, 551-555 (2009) |
| [13] | Chu, MT; Golub, GH, Inverse Eigenvalue Problems: Theory, Algorithms, and Applications (2005), Oxford, UK: Oxford University Press, Oxford, UK · Zbl 1075.65058 · doi:10.1093/acprof:oso/9780198566649.001.0001 |
| [14] | Deng, KK; Peng, Z.; Chen, JL, Sparse probabilistic Boolean network problems: A partial proximal-type operator splitting method, Journal of Industrial & Management Optimization, 15, 1881-1896 (2019) · Zbl 1438.90059 · doi:10.3934/jimo.2018127 |
| [15] | Gu, JW; Ching, WK; Siu, TK; Zheng, H., On modeling credit defaults: a probabilistic Boolean network approach, Risk and Decision Analysis, 4, 119-129 (2013) · Zbl 1294.91182 · doi:10.3233/RDA-2012-0086 |
| [16] | Hastie, T.; Tibshirani, R.; Wainwright, M., Statistical Learning with Sparsity: the Lasso and Generalizations (2015), Boca Raton, FL: CRC Press, Boca Raton, FL · Zbl 1319.68003 · doi:10.1201/b18401 |
| [17] | Iordache, M., Bioucas-Dias, J., Plaza, A.: Unmixing sparse hyperspectral mixtures. 2009 IEEE International Geoscience and Remote Sensing Symposium, pp. IV-85-IV-88 (2009) |
| [18] | Iordache, M.; Bioucas-Dias, JM; Plaza, A., Total variation spatial regularization for sparse hyperspectral unmixing, IEEE Trans. Geosci. Remote Sens., 50, 11, 4484-4502 (2012) · doi:10.1109/TGRS.2012.2191590 |
| [19] | Li, J., Bioucas-Dias, J. M.: Minimum volume simplex analysis: A fast algorithm to unmix hyperspectral data. In: 2008 IEEE International Geoscience and Remote Sensing Symposium, pp. III-250-III-253 (2008) |
| [20] | Lin, MX; Liu, Y-J; Sun, DF; Toh, K-C, Efficient sparse semismooth Newton methods for the clustered Lasso problem, SIAM J. Optim., 29, 2026-2052 (2019) · Zbl 1427.90200 · doi:10.1137/18M1207752 |
| [21] | Mordukhovich, BS, Variational Analysis and Generalized Differentiation I-Basic Theory (2006), Berlin: Springer, Berlin · doi:10.1007/3-540-31246-3 |
| [22] | Moussaoui, S., Idier, J., Chouzenoux, E.: Primal dual interior point optimization for penalized least squares estimation of abundance maps in hyperspectral imaging. In: 2012 4th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS), pp. 1-4, (2012) |
| [23] | Ortega, JM; Rheinboldt, WC, Iterative Solution of Nonlinear Equations in Several Variables (1970), New York: Academic Press, New York · Zbl 0241.65046 |
| [24] | Rockafellar, RT; Wets, R., J-B: Variational Analysis (1998), Berlin: Springer, Berlin · Zbl 0888.49001 · doi:10.1007/978-3-642-02431-3 |
| [25] | Rockafellar, RT, Convex Analysis (1970), Princeton: Princeton University Press, Princeton · Zbl 0932.90001 · doi:10.1515/9781400873173 |
| [26] | Salehani, Y. E., Gazor, S., Kim, I., Yousefi, S.: Sparse hyperspectral unmixing via arctan approximation of L0 norm. In: 2014 IEEE Geoscience and Remote Sensing Symposium, pp. 2930-2933 (2014) |
| [27] | Wen, YW; Wang, M.; Cao, ZY; Cheng, XQ; Ching, WK; Vassiliadis, VS, Sparse solution of nonnegative least squares problems with applications in the construction of probabilistic Booelan networks, Numer. Linear Algebra Appl., 22, 883-899 (2015) · Zbl 1349.65140 · doi:10.1002/nla.2001 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
