×

Elastic-net regularization: iterative algorithms and asymptotic behavior of solutions. (English) Zbl 1213.47011

The goal of the paper is to yield progress in elastic-net regularization coming from statistics introduced and established in the inverse problems community by H. Zou and T. Hastie [“Regularization and variable selection via the elastic net”, J. R. Stat. Soc., Ser. B, Stat. Methodol. 67, No. 2, 301–320 (2005; Zbl 1069.62054)], B.-T. Jin, D. A. Lorenz and S. Schiffler [“Elastic-net regularization: error estimates and active set methods”, Inverse Probl. 25, No. 11, Article ID 115022 (2009; Zbl 1188.49026)], and C. De Mol, E. De Vito and L. Rosasco [“Elastic-net regularization in learning theory”, J. Complexity 25, No. 2, 201–230 (2009; Zbl 1319.62087)]. For an ill-posed linear operator equation \(Kx=y\) with a smoothing forward operator \(K:\ell^2(\Gamma) \to H\) (with \(H\) a Hilbert space), motivated by imaging problems and sparsity effects, penalized least squares with penality term \(p_\omega(x)=2 \omega \|x\|_{\ell^1}+\|x\|_{\ell^2}^2\) are exploited for obtaining stable approximate solutions. The used linear combination of \(\ell^1\) and \(\ell^2\) terms with weight parameter \(\omega>0\) can be adapted to different situations in a wide range of applications. In contrast to the standard case of \(\ell^2\) regularization, the handling of \(\ell^1\) terms requires special numerical algorithms for computing the regularized solutions. On the one hand, this paper makes contributions to such algorithms. On the other hand, the paper contains functional analytic studies on the convergence and on the asymptotic behaviour of regularized solutions. The authors discuss the interplay of balls having radius \(R>0\) in \(\ell^1\) and the magnitude of \(\omega\), which means that the choice of regularization parameters as always is a delicate problem. It seems to be an aim of the authors to cast in a unified framework the classical Tikhonov regularization, the \(\ell^1\) regularization, and the mixed version under consideration.

MSC:

47A52 Linear operators and ill-posed problems, regularization
65J20 Numerical solutions of ill-posed problems in abstract spaces; regularization
49K40 Sensitivity, stability, well-posedness
65F22 Ill-posedness and regularization problems in numerical linear algebra
65C60 Computational problems in statistics (MSC2010)
Full Text: DOI

References:

[1] Engl H.W., Regularization of Inverse Problems (1996) · Zbl 0859.65054
[2] Dontchev A.L., Well-posed Optimization Problems (1993) · Zbl 0797.49001 · doi:10.1007/BFb0084195
[3] DOI: 10.1109/TIT.2005.862083 · Zbl 1231.94017 · doi:10.1109/TIT.2005.862083
[4] DOI: 10.1109/TIT.2006.871582 · Zbl 1288.94016 · doi:10.1109/TIT.2006.871582
[5] Tibshirani R., J. Roy. Statist. Soc. Series B 56 pp 267–
[6] DOI: 10.1007/s00041-008-9035-z · Zbl 1175.94060 · doi:10.1007/s00041-008-9035-z
[7] DOI: 10.1002/cpa.20303 · Zbl 1202.65046 · doi:10.1002/cpa.20303
[8] DOI: 10.1016/j.acha.2009.02.003 · Zbl 1170.65318 · doi:10.1016/j.acha.2009.02.003
[9] DOI: 10.1088/0266-5611/25/3/035008 · Zbl 1162.65333 · doi:10.1088/0266-5611/25/3/035008
[10] DOI: 10.1111/j.1467-9868.2005.00503.x · Zbl 1069.62054 · doi:10.1111/j.1467-9868.2005.00503.x
[11] DOI: 10.1088/0266-5611/25/11/115022 · Zbl 1188.49026 · doi:10.1088/0266-5611/25/11/115022
[12] DOI: 10.1016/j.jco.2009.01.002 · Zbl 1319.62087 · doi:10.1016/j.jco.2009.01.002
[13] DOI: 10.1089/cmb.2008.0171 · doi:10.1089/cmb.2008.0171
[14] DOI: 10.1007/s10287-008-0070-7 · Zbl 1168.62301 · doi:10.1007/s10287-008-0070-7
[15] DOI: 10.1137/050626090 · Zbl 1179.94031 · doi:10.1137/050626090
[16] DOI: 10.1007/s00041-008-9041-1 · Zbl 1175.65061 · doi:10.1007/s00041-008-9041-1
[17] Polyak B.T., Introduction to Optimization. Optimization Software, Inc. (1987) · Zbl 0708.90083
[18] DOI: 10.1007/s00041-008-9039-8 · Zbl 1175.65062 · doi:10.1007/s00041-008-9039-8
[19] Ivanov V.V., The Theory of Approximate Methods and their Application to the Numerical Solution of Singular Integral Equations (1976) · Zbl 0346.65065
[20] DOI: 10.1515/JIIP.2008.025 · Zbl 1161.65041 · doi:10.1515/JIIP.2008.025
[21] Ciarlet P.G., Introduction to Numerical Linear Algebra and Optimisation (1989)
[22] DOI: 10.1137/0716071 · Zbl 0426.65050 · doi:10.1137/0716071
[23] DOI: 10.1137/S1052623495290179 · Zbl 0876.49009 · doi:10.1137/S1052623495290179
[24] DOI: 10.1002/cpa.20042 · Zbl 1077.65055 · doi:10.1002/cpa.20042
[25] DOI: 10.1088/0266-5611/24/5/055020 · Zbl 1157.65033 · doi:10.1088/0266-5611/24/5/055020
[26] Ramlau R., Electron. Trans. Numer. Anal. 30 pp 54– (2008)
[27] DOI: 10.1007/s00607-007-0245-z · Zbl 1147.68790 · doi:10.1007/s00607-007-0245-z
[28] Ekeland I., Infinite-Dimensional Optimization and Convexity (1983) · Zbl 0565.49003
[29] Brezis H., Analyse Fonctionnelle (1983)
[30] Duchi J., Proceeding of the 5th International Conference on Machine Learning 307 (2008)
[31] DOI: 10.1137/S1064827596304010 · Zbl 0919.94002 · doi:10.1137/S1064827596304010
[32] DOI: 10.1214/009053606000001523 · Zbl 1139.62019 · doi:10.1214/009053606000001523
[33] DOI: 10.1137/070698920 · Zbl 1180.65076 · doi:10.1137/070698920
[34] DOI: 10.1089/cmb.2008.0171 · doi:10.1089/cmb.2008.0171
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.