Elastic-net regularization: iterative algorithms and asymptotic behavior of solutions. (English) Zbl 1213.47011
The goal of the paper is to yield progress in elastic-net regularization coming from statistics introduced and established in the inverse problems community by H. Zou and T. Hastie [“Regularization and variable selection via the elastic net”, J. R. Stat. Soc., Ser. B, Stat. Methodol. 67, No. 2, 301–320 (2005; Zbl 1069.62054)], B.-T. Jin, D. A. Lorenz and S. Schiffler [“Elastic-net regularization: error estimates and active set methods”, Inverse Probl. 25, No. 11, Article ID 115022 (2009; Zbl 1188.49026)], and C. De Mol, E. De Vito and L. Rosasco [“Elastic-net regularization in learning theory”, J. Complexity 25, No. 2, 201–230 (2009; Zbl 1319.62087)]. For an ill-posed linear operator equation \(Kx=y\) with a smoothing forward operator \(K:\ell^2(\Gamma) \to H\) (with \(H\) a Hilbert space), motivated by imaging problems and sparsity effects, penalized least squares with penality term \(p_\omega(x)=2 \omega \|x\|_{\ell^1}+\|x\|_{\ell^2}^2\) are exploited for obtaining stable approximate solutions. The used linear combination of \(\ell^1\) and \(\ell^2\) terms with weight parameter \(\omega>0\) can be adapted to different situations in a wide range of applications. In contrast to the standard case of \(\ell^2\) regularization, the handling of \(\ell^1\) terms requires special numerical algorithms for computing the regularized solutions. On the one hand, this paper makes contributions to such algorithms. On the other hand, the paper contains functional analytic studies on the convergence and on the asymptotic behaviour of regularized solutions. The authors discuss the interplay of balls having radius \(R>0\) in \(\ell^1\) and the magnitude of \(\omega\), which means that the choice of regularization parameters as always is a delicate problem. It seems to be an aim of the authors to cast in a unified framework the classical Tikhonov regularization, the \(\ell^1\) regularization, and the mixed version under consideration.
Reviewer: Bernd Hofmann (Chemnitz)
MSC:
47A52 | Linear operators and ill-posed problems, regularization |
65J20 | Numerical solutions of ill-posed problems in abstract spaces; regularization |
49K40 | Sensitivity, stability, well-posedness |
65F22 | Ill-posedness and regularization problems in numerical linear algebra |
65C60 | Computational problems in statistics (MSC2010) |
Keywords:
linear ill-posed problems; regularization; sparsity; imaging; elastic-net; splitting algorithmReferences:
[1] | Engl H.W., Regularization of Inverse Problems (1996) · Zbl 0859.65054 |
[2] | Dontchev A.L., Well-posed Optimization Problems (1993) · Zbl 0797.49001 · doi:10.1007/BFb0084195 |
[3] | DOI: 10.1109/TIT.2005.862083 · Zbl 1231.94017 · doi:10.1109/TIT.2005.862083 |
[4] | DOI: 10.1109/TIT.2006.871582 · Zbl 1288.94016 · doi:10.1109/TIT.2006.871582 |
[5] | Tibshirani R., J. Roy. Statist. Soc. Series B 56 pp 267– |
[6] | DOI: 10.1007/s00041-008-9035-z · Zbl 1175.94060 · doi:10.1007/s00041-008-9035-z |
[7] | DOI: 10.1002/cpa.20303 · Zbl 1202.65046 · doi:10.1002/cpa.20303 |
[8] | DOI: 10.1016/j.acha.2009.02.003 · Zbl 1170.65318 · doi:10.1016/j.acha.2009.02.003 |
[9] | DOI: 10.1088/0266-5611/25/3/035008 · Zbl 1162.65333 · doi:10.1088/0266-5611/25/3/035008 |
[10] | DOI: 10.1111/j.1467-9868.2005.00503.x · Zbl 1069.62054 · doi:10.1111/j.1467-9868.2005.00503.x |
[11] | DOI: 10.1088/0266-5611/25/11/115022 · Zbl 1188.49026 · doi:10.1088/0266-5611/25/11/115022 |
[12] | DOI: 10.1016/j.jco.2009.01.002 · Zbl 1319.62087 · doi:10.1016/j.jco.2009.01.002 |
[13] | DOI: 10.1089/cmb.2008.0171 · doi:10.1089/cmb.2008.0171 |
[14] | DOI: 10.1007/s10287-008-0070-7 · Zbl 1168.62301 · doi:10.1007/s10287-008-0070-7 |
[15] | DOI: 10.1137/050626090 · Zbl 1179.94031 · doi:10.1137/050626090 |
[16] | DOI: 10.1007/s00041-008-9041-1 · Zbl 1175.65061 · doi:10.1007/s00041-008-9041-1 |
[17] | Polyak B.T., Introduction to Optimization. Optimization Software, Inc. (1987) · Zbl 0708.90083 |
[18] | DOI: 10.1007/s00041-008-9039-8 · Zbl 1175.65062 · doi:10.1007/s00041-008-9039-8 |
[19] | Ivanov V.V., The Theory of Approximate Methods and their Application to the Numerical Solution of Singular Integral Equations (1976) · Zbl 0346.65065 |
[20] | DOI: 10.1515/JIIP.2008.025 · Zbl 1161.65041 · doi:10.1515/JIIP.2008.025 |
[21] | Ciarlet P.G., Introduction to Numerical Linear Algebra and Optimisation (1989) |
[22] | DOI: 10.1137/0716071 · Zbl 0426.65050 · doi:10.1137/0716071 |
[23] | DOI: 10.1137/S1052623495290179 · Zbl 0876.49009 · doi:10.1137/S1052623495290179 |
[24] | DOI: 10.1002/cpa.20042 · Zbl 1077.65055 · doi:10.1002/cpa.20042 |
[25] | DOI: 10.1088/0266-5611/24/5/055020 · Zbl 1157.65033 · doi:10.1088/0266-5611/24/5/055020 |
[26] | Ramlau R., Electron. Trans. Numer. Anal. 30 pp 54– (2008) |
[27] | DOI: 10.1007/s00607-007-0245-z · Zbl 1147.68790 · doi:10.1007/s00607-007-0245-z |
[28] | Ekeland I., Infinite-Dimensional Optimization and Convexity (1983) · Zbl 0565.49003 |
[29] | Brezis H., Analyse Fonctionnelle (1983) |
[30] | Duchi J., Proceeding of the 5th International Conference on Machine Learning 307 (2008) |
[31] | DOI: 10.1137/S1064827596304010 · Zbl 0919.94002 · doi:10.1137/S1064827596304010 |
[32] | DOI: 10.1214/009053606000001523 · Zbl 1139.62019 · doi:10.1214/009053606000001523 |
[33] | DOI: 10.1137/070698920 · Zbl 1180.65076 · doi:10.1137/070698920 |
[34] | DOI: 10.1089/cmb.2008.0171 · doi:10.1089/cmb.2008.0171 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.