Nonparametric regression with correlated errors. (English) Zbl 1059.62537
Summary: Nonparametric regression techniques are often sensitive to the presence of correlation in the errors. The practical consequences of this sensitivity are explained, including the breakdown of several popular data-driven smoothing parameter selection methods. We review the existing literature in kernel regression, smoothing splines and wavelet regression under correlation, both for short-range and long-range dependence. Extensions to random design, higher dimensional models and adaptive estimation are discussed.
Keywords:
Kernel regression; splines; wavelet regression; adaptive estimation; smoothing parameter selectionReferences:
| [1] | Adenstedt, R. K. (1974). On large sample estimation for the mean of a stationarysequence. Ann. Statist. 2 1095-1107. · Zbl 0296.62081 · doi:10.1214/aos/1176342867 |
| [2] | Altman, N. (1994). Krige, smooth, both or neither? Technical report, Biometrics Unit, Cornell Univ. · Zbl 1016.62032 |
| [3] | Altman, N. S. (1990). Kernel smoothing of data with correlated errors. J. Amer. Statist. Assoc. 85 749-759. JSTOR: · doi:10.2307/2290011 |
| [4] | Aronszajn, N. (1950). Theoryof reproducing kernels. Trans. Amer. Math. Soc. 68 337-404. JSTOR: · Zbl 0037.20701 · doi:10.2307/1990404 |
| [5] | Barron, A. R. and Barron, R. L. (1988). Statistical learning networks: a unifying view. In Computer Science and Statistics: Proceedings of the 21st Interface, 192-203. |
| [6] | Beran, J. (1992). Statistical methods for data with long-range dependence. Statist. Sci. 7 404-416. |
| [7] | Beran, J. (1994). Statistics for Long-Memory Processes. Chapman and Hall, New York. · Zbl 0869.60045 |
| [8] | Bierens, H. (1983). Uniform consistencyof kernel estimators of a regression function under generalized conditions. J. Amer. Statist. Assoc. 78 699-707. Breiman, L., Friedman, J. H., Olshen, R. and Stone, C. J. JSTOR: · Zbl 0565.62027 · doi:10.2307/2288140 |
| [9] | . Classification and Regression Trees. Wadsworth, Belmont, CA. · Zbl 0541.62042 |
| [10] | Chiu, S.-T. (1989). Bandwidth selection for kernel estimate with correlated noise. Statist. Probab. Lett. 8 347-354. · Zbl 0676.62040 · doi:10.1016/0167-7152(89)90043-6 |
| [11] | Chu, C.-K. and Marron, J. S. (1991). Comparison of two bandwidth selectors with dependent errors. Ann. Statist. 19 1906-1918. · Zbl 0738.62042 · doi:10.1214/aos/1176348377 |
| [12] | Collomb, G. and Härdle, W. (1986). Strong uniform convergence rates in robust nonparametric time series analysis. Stochastic Process. Appl. 23 77-89. · Zbl 0612.62127 · doi:10.1016/0304-4149(86)90017-7 |
| [13] | Cox, D. R. (1984). Long-range dependence: a review. In Statistics: An Appraisal. Proceedings 50th Anniversary Conference, (H. A. David and H. T. David, eds.) 55-74. Iowa State Univ. Press. |
| [14] | Diggle, P. J. and Hutchinson, M. F. (1989). On spline smoothing with autocorrelated errors. Austral. J. Statist. 31 166-182. · Zbl 0707.62080 · doi:10.1111/j.1467-842X.1989.tb00510.x |
| [15] | Donoho, D. L. and Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist. 26 879-921. · Zbl 0935.62041 · doi:10.1214/aos/1024691081 |
| [16] | Efromovich, S. (1999). How to overcome curse of long-memory errors in nonparametric regression. IEEE Trans. Inform. Theory 45, 1735-1741. · Zbl 0959.62038 · doi:10.1109/18.771257 |
| [17] | Friedman, J. and Stuetzle, W. (1981). Projection pursuit regression. J. Amer. Statist. Assoc. 76 817-823. JSTOR: · doi:10.2307/2287576 |
| [18] | Gu, C. and Wahba, G. (1993). Semiparametric ANOVA with tensor product thin plate spline. J. Roy. Statist. Assoc. Ser. B 55 353-368. JSTOR: · Zbl 0786.62048 |
| [19] | Hall, P. and Hart, J. D. (1990). Nonparametric regression with long-range dependence. Stochastic Process. Appl. 36 339-351. · Zbl 0713.62048 · doi:10.1016/0304-4149(90)90100-7 |
| [20] | Hall, P. Lahiri, S. N. and Polzehl, J. (1995). On bandwidth choice in nonparametric regression with both shortand long-range dependent errors. Ann. Statist. 23 1921-1936. · Zbl 0856.62041 · doi:10.1214/aos/1034713640 |
| [21] | Härdle, W., Hall, P. and Marron, J. S. (1988). How far are automaticallychosen regression smoothing parameters from their optimum? J. Amer. Statist. Assoc. 83 86-95. JSTOR: · Zbl 0644.62048 · doi:10.2307/2288922 |
| [22] | Härdle, W., Hall, P. and Marron, J. S. (1992). Regression smoothing parameters that are not far from their optimum. J. Amer. Statist. Assoc. 87 227-233. JSTOR: · Zbl 0850.62352 · doi:10.2307/2290473 |
| [23] | Hart, D. (1997). Nonparametric Smoothing and Lack-of-Fit Tests. Springer, New York. · Zbl 0886.62043 |
| [24] | Hart, J. D. (1991). Kernel regression estimation with time series errors. J. Roy. Statist. Assoc. Ser. B 53 173-187. JSTOR: · Zbl 0800.62215 |
| [25] | Hart, J. D. (1994). Automated kernel smoothing of dependent data byusing time series cross-validation. J. Roy. Statist. Assoc. Ser. B 56 529-542. JSTOR: · Zbl 0800.62224 |
| [26] | Harville, D. (1976). Extension of the Gauss-Markov theorem to include the estimation of random effects. Ann. Statist. 4 384-395. · Zbl 0323.62043 · doi:10.1214/aos/1176343414 |
| [27] | Herrmann, E., Gasser, T. and Kneip, A. (1992). Choice of bandwidth for kernel regression when residuals are correlated. Biometrika 79 783-795. JSTOR: · Zbl 0765.62044 · doi:10.1093/biomet/79.4.783 |
| [28] | Johnstone, I. and Silverman, B. (1997). Wavelet threshold estimators for data with correlated noise. J. Roy. Statist. Assoc. Ser. B 59 319-351. JSTOR: · Zbl 0886.62044 · doi:10.1111/1467-9868.00071 |
| [29] | Kimeldorf, G. S. and Wahba, G. (1971). Some results on Tchebycheffian spline functions. J. Math. Anal. Appl. 33 82-94. · Zbl 0201.39702 · doi:10.1016/0022-247X(71)90184-3 |
| [30] | Kohn, R., Ansley, C. F. and Tharm, D. (1991). The performance of cross-validation and maximum likelihood estimators of spline smoothing parameters. J. Amer. Statist. Assoc. 86 1042-1050. JSTOR: · Zbl 0850.62351 · doi:10.2307/2290523 |
| [31] | Kohn, R., Ansley, C. F. and Wong, C. (1992). Nonparametric spline regression with autoregressive moving average errors. Biometrika 79 335-346. JSTOR: · Zbl 0751.62017 · doi:10.1093/biomet/79.2.335 |
| [32] | K ünsch, H., Beran, J. and Hampel, F. (1993). Contrasts under long-range correlations. Ann. Statist. 21 943-964. · Zbl 0795.62077 · doi:10.1214/aos/1176349159 |
| [33] | Laslett, G. (1994). Kriging and splines: an empirical comparison of their predictive performance in some applications. J. Amer. Statist. Assoc. 89 392-409. JSTOR: · doi:10.2307/2290837 |
| [34] | Nason, G. P. (1996). Wavelet shrinkage using cross-validation. J. Roy. Statist. Soc. Ser. B 58 463-479. JSTOR: · Zbl 0853.62034 |
| [35] | Nicoleris, T. and Yatracos, Y. G. (1997). Rate of convergence of estimates, Kolmogrov’s entropyand the dimensionalityreduction principle in regression. Ann. Statist. 25 2493-2511. · Zbl 0909.62063 · doi:10.1214/aos/1030741082 |
| [36] | Opsomer, J.-D. (1995). Estimating a function bylocal linear regression when the errors are correlated. Preprint 95-42, Dept. Statistics, Iowa State Univ. |
| [37] | Opsomer, J.-D. (1997). Nonparametric regression in the presence of correlated errors. In Modelling Longitudinal and Spatially Correlated Data: Methods, Applications and Future Directions (T. Gregoire, D. Brillinger, P. Diggle, E. RussekCohen, W. Warren and R. Wolfinger, eds.) 339-348. Springer, New York. · Zbl 0897.62039 |
| [38] | Opsomer, J.-D. and Ruppert, D. (1997). Fitting a bivariate additive model bylocal polynomial regression. Ann. Statist. 25 186-211. · Zbl 0869.62026 · doi:10.1214/aos/1034276626 |
| [39] | Opsomer, J.-D. and Ruppert, D. (1998). A fullyautomated bandwidth selection method for fitting additive models by local polynomial regression. J. Amer. Statist. Assoc. 93 605-619. JSTOR: · Zbl 0953.62034 · doi:10.2307/2670112 |
| [40] | Priestley, M. B. and Chao, M. T. (1972). Nonparametric function fitting. J. Roy. Statist. Assoc. Ser. B 34 385-392. JSTOR: · Zbl 0263.62044 |
| [41] | Ruppert, D., Sheather, S. J. and Wand, M. P. (1995). An effective bandwidth selector for local least squares regression. J. Amer. Statist. Assoc. 90 1257-1270. JSTOR: · Zbl 0868.62034 · doi:10.2307/2291516 |
| [42] | Ruppert, D. and Wand, M. P. (1994). Multivariate locally weighted least squares regression. Ann. Statist. 22 1346-1370. · Zbl 0821.62020 · doi:10.1214/aos/1176325632 |
| [43] | Stone, C. J. (1994). The use of polynomial splines and their tensor products in multivariate function estimation. Ann. Statist. 22 118-184. Stone, C. J. Hansen, M. H. Kooperberg, C. and Truong, Y. · Zbl 0827.62038 · doi:10.1214/aos/1176325361 |
| [44] | . Polynomial splines and their tensor products in extended linear modeling (with discussion). Ann. Statist. 25 1371-1470. · Zbl 0924.62036 · doi:10.1214/aos/1031594728 |
| [45] | Wahba, G. (1978). Improper priors, spline smoothing, and the problem of guarding against model errors in regression. J. Roy. Statist. Assoc. Ser. B 40 364-372. JSTOR: · Zbl 0407.62048 |
| [46] | Wahba, G. (1983). Bayesian confidence intervals for the crossvalidated smoothing spline. J. Roy. Statist. Assoc. Ser. B 45 133-150. JSTOR: · Zbl 0538.65006 |
| [47] | Wahba, G. (1985). A comparison of GCV and GML for choosing the smoothing parameters in the generalized spline smoothing problem. Ann. Statist. 4 1378-1402. · Zbl 0596.65004 · doi:10.1214/aos/1176349743 |
| [48] | Wahba, G. (1990). Spline Models for Observational Data. SIAM, Philadelphia. · Zbl 0813.62001 |
| [49] | Wahba, G. and Wang, Y. (1993). Behavior near zero of the distribution of GCV smoothing parameter estimates for splines. Statist. Probab. Lett. 25 105-111. · Zbl 0838.62028 · doi:10.1016/0167-7152(94)00211-P |
| [50] | Wahba, G. Wang, Y., Gu, C., Klein, R. and Klein, B. (1995). Smoothing spline ANOVA for exponential families, with application to the Wisconsin Epidemiological Studyof Diabetic Retinopathy. Ann. Statist. 23 1865-1895. · Zbl 0854.62042 · doi:10.1214/aos/1034713638 |
| [51] | Wang, Y. (1996). Function estimation via wavelet shrinkage for long-memorydata. Ann. Statist. 24 466-484. Wang, Y. (1998a). Mixed-effects smoothing spline ANOVA. J. Roy. Statist. Assoc. Ser. B 60 159-174. Wang, Y. (1998b). Smoothing spline models with correlated random errors. J. Amer. Statist. Assoc. 93 341-348. · Zbl 0859.62042 · doi:10.1214/aos/1032894449 |
| [52] | Yang, Y. (1997). Nonparametric regression with dependent errors. Preprint 97-29, Dept. Statistics, Iowa State Univ. (A shorter version is accepted in Bernoulli.) · Zbl 1006.62041 · doi:10.2307/3318730 |
| [53] | Yang, Y. (1999). Model selection for nonparametric regression. Statist. Sinica 9 475-499. · Zbl 0921.62051 |
| [54] | Yang, Y. (2000). Combining different procedures for adaptive regression. J. Multivariate Anal. 74 135-161. · Zbl 0964.62032 · doi:10.1006/jmva.1999.1884 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
