×

On stepwise pattern recovery of the fused Lasso. (English) Zbl 1468.62161

Summary: We study the property of the Fused Lasso Signal Approximator (FLSA) for estimating a blocky signal sequence with additive noise. We transform the FLSA to an ordinary Lasso problem, and find that in general the resulting design matrix does not satisfy the irrepresentable condition that is known as an almost necessary and sufficient condition for exact pattern recovery. We give necessary and sufficient conditions on the expected signal pattern such that the irrepresentable condition holds in the transformed Lasso problem. However, these conditions turn out to be very restrictive. We apply the newly developed preconditioning method – Puffer Transformation [J. Jia and K. Rohe, Electron. J. Stat. 9, No. 1, 1150–1172 (2015; Zbl 1321.62083)] to the transformed Lasso and call the new procedure the preconditioned fused Lasso. We give non-asymptotic results for this method, showing that as long as the signal-to-noise ratio is not too small, our preconditioned fused Lasso estimator always recovers the correct pattern with high probability. Theoretical results give insight into what controls the ability of recovering the pattern – it is the noise level instead of the length of the signal sequence. Simulations further confirm our theorems and visualize the significant improvement of the preconditioned fused Lasso estimator over the vanilla FLSA in exact pattern recovery.

MSC:

62-08 Computational methods for problems pertaining to statistics
62J07 Ridge regression; shrinkage estimators (Lasso)

Citations:

Zbl 1321.62083

References:

[1] Cadima, J.; Jolliffe, I., On relationships between uncentred and column-centred principal component analysis, Pak. J. Stat., 25, 4, 473-503, (2009) · Zbl 1509.62265
[2] Cai, J.; Candes, E.; Shen, Z., A singular value thresholding algorithm for matrix completion, SIAM J. Optim., 20, 4, 1956-1982, (2010) · Zbl 1201.90155
[3] Donoho, D. L.; Elad, M.; Temlyakov, V. N., Stable recovery of sparse overcomplete representations in the presence of noise, IEEE Trans. Inf. Theory, 52, 1, 6-18, (2006) · Zbl 1288.94017
[4] Fan, J.; Li, R., Variable selection via nonconcave penalized likelihood and its oracle properties, J. Amer. Statist. Assoc., 96, 456, 1348-1360, (2001) · Zbl 1073.62547
[5] Friedman, J.; Hastie, T.; Höfling, H.; Tibshirani, R., Pathwise coordinate optimization, Ann. Appl. Stat., 1, 2, 302-332, (2007) · Zbl 1378.90064
[6] Greenshtein, E.; Ritov, Y. A., Persistence in high-dimensional linear predictor selection and the virtue of overparametrization, Bernoulli, 10, 6, 971-988, (2004) · Zbl 1055.62078
[7] Harchaoui, Zaıd; Lévy-Leduc, Céline, Catching change-points with lasso, Adv. Neural Inf. Process. Syst., 20, 161-168, 18, (2008)
[8] Harchaoui, Zaıd; Lévy-Leduc, Céline, Multiple change-point estimation with a total variation penalty, J. Amer. Statist. Assoc., 105, 492, 1480-1493, (2010) · Zbl 1388.62211
[9] Hoefling, H., A path algorithm for the fused lasso signal approximator, J. Comput. Graph. Statist., 19, 4, 984-1006, (2010)
[10] Jia, J.; Rohe, K., Preconditioning the lasso for sign consistency, Electron. J. Stat., 9, 1150-1172, (2015) · Zbl 1321.62083
[11] Jia, J.; Rohe, K.; Yu, B., The lasso under heteroscedasticity, Statist. Sinica, 23, 99-118, (2013) · Zbl 1259.62042
[12] Knight, K.; Fu, W., Asymptotics for lasso-type estimators, Ann. Statist., 1356-1378, (2000) · Zbl 1105.62357
[13] Meinshausen, N.; Bühlmann, P., High-dimensional graphs and variable selection with the lasso, Ann. Statist., 34, 3, 1436-1462, (2006) · Zbl 1113.62082
[14] Rinaldo, A., Properties and refinements of the fused lasso, Ann. Statist., 37, 5B, 2922-2952, (2009) · Zbl 1173.62027
[15] Sharpnack, James, Rinaldo, Alessandro, Singh, Aarti, 2012. Sparsistency of the edge lasso over graphs. In: International Conference on Artificial Intelligence and Statistics (AISTATS, JMLR: W&CP), volume 22, pp. 1028-1036.
[16] Tibshirani, R., Regression shrinkage and selection via the lasso, J. R. Stat. Soc. Ser. B Stat. Methodol., 267-288, (1996) · Zbl 0850.62538
[17] Tibshirani, R.; Saunders, M.; Rosset, S.; Zhu, J.; Knight, K., Sparsity and smoothness via the fused lasso, J. R. Stat. Soc. Ser. B Stat. Methodol., 67, 1, 91-108, (2004) · Zbl 1060.62049
[18] Tibshirani, Ryan J.; Taylor, Jonathan, The solution path of the generalized lasso, Ann. Statist., 39, 3, 1335-1371, (2011) · Zbl 1234.62107
[19] Tibshirani, R.; Wang, P., Spatial smoothing and hot spot detection for cgh data using the fused lasso, Biostatistics, 9, 1, 18-29, (2008) · Zbl 1274.62886
[20] Tropp, J. A., Just relax: convex programming methods for identifying sparse signals in noise, IEEE Trans. Inf. Theory, 52, 3, 1030-1051, (2006) · Zbl 1288.94025
[21] Wainwright, M. J., Information-theoretic limits on sparsity recovery in the high-dimensional and noisy setting, IEEE Trans. Inf. Theory, 55, 12, 5728-5741, (2009) · Zbl 1367.94106
[22] Xu, C., Liu, T., Tao, D., Xu, C., 2014. Local rademacher complexity for multi-label learning. arXiv preprint arXiv:1410.6990.
[23] Xu, C.; Tao, D.; Xu, C., Multi-view intact space learning, IEEE Trans. Pattern Anal. Mach. Intell., 1, 1, 1, (2015)
[24] Zhang, C. H.; Huang, J., The sparsity and bias of the lasso selection in high-dimensional linear regression, Ann. Statist., 36, 4, 1567-1594, (2008) · Zbl 1142.62044
[25] Zhao, P.; Yu, B., On model selection consistency of lasso, J. Mach. Learn. Res., 7, 2, 2541, (2006) · Zbl 1222.62008
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.