×

Robust GCV choice of the regularization parameter for correlated data. (English) Zbl 1217.65097

Author’s abstract: We consider Tikhonov regularization of linear inverse problems with discrete noisy data containing correlated errors. Generalized cross-validation (GCV) is a prominent parameter choice method, but it is known to perform poorly if the sample size \(n\) is small or if the errors are correlated, sometimes giving the extreme value 0. We explain why this can occur and show that the robust GCV methods perform better. In particular, it is shown that, for any data set, there is a value of the robustness parameter below which the strong robust GCV method (R\(_1\)GCV) will not choose the value 0. We also show that if the errors are correlated with a certain covariance model then, for a range of values of the unknown correlation parameter, the “expected” R\(_1\)GCV estimate has a near optimal rate as \( n \rightarrow \infty\). Numerical results for the problem of second derivative estimation are consistent with the theoretical results and show that R\(_1\)GCV gives reliable and accurate estimates.

MSC:

65J10 Numerical solutions to equations with linear operators
65J20 Numerical solutions of ill-posed problems in abstract spaces; regularization
65J22 Numerical solution to inverse problems in abstract spaces
47A52 Linear operators and ill-posed problems, regularization
65R30 Numerical methods for ill-posed problems for integral equations
45B05 Fredholm integral equations
Full Text: DOI

References:

[1] F. Bauer and M.A. Lukas, Comparing parameter choice methods for regularization of ill-posed problems , submitted (2010). · Zbl 1220.65063
[2] F. Bauer, P. Mathé and S. Pereverzev, Local solutions to inverse problems in geodesy. The impact of the noise covariance structure upon the accuracy of estimation , J. Geod. 81 (2007), 39-51. · Zbl 1138.86006 · doi:10.1007/s00190-006-0049-5
[3] P. Craven and G. Wahba, Smoothing noisy data with spline functions , Numer. Math. 31 (1979), 377-403. · Zbl 0377.65007 · doi:10.1007/BF01404567
[4] D.J. Cummins, T.G. Filloon and D. Nychka, Confidence intervals for nonparametric curve estimates : Toward more uniform pointwise coverage , J. Amer. Statist. Assoc. 96 (2001), 233-246. · Zbl 1015.62049 · doi:10.1198/016214501750332811
[5] B. Efron, Selection criteria for scatterplot smoothers , Ann. Statist. 29 (2001), 470-504. · Zbl 1012.62040 · doi:10.1214/aos/1009210549
[6] C. Han and C. Gu, Optimal smoothing with correlated data , Sankhyā 70 · Zbl 1193.62066
[7] P.C. Hansen, Regularization tools : A Matlab package for analysis and solution of discrete ill-posed problems , Numer. Algorithms 6 (1992), 1-35. · Zbl 0789.65029 · doi:10.1007/BF02149761
[8] ——–, Rank-deficient and discrete ill-posed problems , SIAM, Philadelphia, 1998. · Zbl 0890.65037 · doi:10.1137/1.9780898719697
[9] M. Jansen and A. Bultheel, Multiple wavelet threshold estimation by generalized cross validation for images with correlated noise , IEEE Trans. Image Process. 8 (1999), 947-953. · Zbl 0952.65112 · doi:10.1109/83.772237
[10] Y.J. Kim and C. Gu, Smoothing spline Gaussian regression : More scalable computation via efficient approximation , J. Roy. Statist. Soc. · Zbl 1062.62067 · doi:10.1046/j.1369-7412.2003.05316.x
[11] S.C. Kou and B. Efron, Smoothers and the \(C_p\), generalized maximum likelihood, and extended exponential criteria : A geometric approach , J. Amer. Statist. Assoc. 97 (2002), 766-782. · Zbl 1048.62044 · doi:10.1198/016214502388618582
[12] K.C. Li, Asymptotic optimality of \(C_L\) and generalized cross-validation in ridge regression with application to spline smoothing , Ann. Statist. 14 (1986), 1101-1112. · Zbl 0629.62043 · doi:10.1214/aos/1176350052
[13] M.A. Lukas, Asymptotic optimality of generalized cross-validation for choosing the regularization parameter , Numer. Math. 66 (1993), 41-66. · Zbl 0791.65037 · doi:10.1007/BF01385687
[14] ——–, Convergence rates for moment collocation solutions of linear operator equations , Numer. Funct. Anal. Optim. 16 (1995), 743-750. · Zbl 0856.65060 · doi:10.1080/01630569508816641
[15] ——–, Comparisons of parameter choice methods for regularization with discrete noisy data , Inverse Problems 14 (1998), 161-184. · Zbl 0904.65053 · doi:10.1088/0266-5611/14/1/014
[16] ——–, Robust generalized cross-validation for choosing the regularization parameter , Inverse Problems 22 (2006), 1883-1902. · Zbl 1104.62032 · doi:10.1088/0266-5611/22/5/021
[17] ——–, Strong robust generalized cross-validation for choosing the regularization parameter , Inverse Problems, 24 , (2008), 034006. · Zbl 1137.62018 · doi:10.1088/0266-5611/24/3/034006
[18] M.A. Lukas, F.R. de Hoog and R.S. Anderssen, Performance of robust GCV and modified GCV for spline smoothing , submitted (2009). · Zbl 1246.62110
[19] D.S. Mitrinović, J.E. Pečarić and A.M. Fink, Classical and new inequalities in analysis , Kluwer, Dordrecht, 1993. · Zbl 0771.26009
[20] J. Opsomer, Y. Wang and Y. Yang, Nonparametric regression with correlated errors , Statist. Sci. 16 (2001), 134-153. · Zbl 1059.62537 · doi:10.1214/ss/1009213287
[21] T. Robinson and R. Moyeed, Making robust the cross-validation choice of smoothing parameter in spline smoothing regression , Comm. Statist. Theory Methods 18 (1989), 523-539. · Zbl 0696.62182 · doi:10.1080/03610928908829916
[22] G.B. Rybicki and W.H. Press, Class of fast methods for processing irregularly sampled or otherwise inhomogeneous one-dimensional data , Phys. Rev. Lett. 74 (1995), 1060-1063.
[23] A.M. Thompson, J.W. Kay and D.M. Titterington, A cautionary note about crossvalidatory choice , J. Stat. Comput. Simul. 33 (1989), 199-216. · Zbl 0726.62120 · doi:10.1080/00949658908811198
[24] R. Vio, P. Ma, W. Zhong, J. Nagy, L. Tenorio and W. Wamsteker, Estimation of regularization parameters in multiple-image deblurring , Astron. Astrophys. 423 (2004), 1179-1186.
[25] G. Wahba, Practical approximate solutions to linear operator equations when the data are noisy , SIAM J. Numer. Anal. 14 (1977), 651-667. · Zbl 0402.65032 · doi:10.1137/0714044
[26] ——–, Spline models for observational data , SIAM, Philadelphia, 1990. · Zbl 0813.62001
[27] G. Wahba and Y.D. Wang, Behavior near zero of the distribution of GCV smoothing parameter estimates , Statist. Probab. Lett. 25 (1995), 105-111. · Zbl 0838.62028 · doi:10.1016/0167-7152(94)00211-P
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.