Adaptive estimation in autoregression or \(\beta\)-mixing regression via model selection. (English) Zbl 1012.62034
Summary: We study the problem of estimating some unknown regression function in a \(\beta\)-mixing dependent framework. To this end, we consider some collection of models which are finite-dimensional spaces. A penalized least-squares estimator (PLSE) is built on a data driven selected model among this collection. We state non-asymptotic risk bounds for this PLSE and give several examples where the procedure can be applied (autoregression, regression with arithmetically \(\beta\)-mixing design points, regression with mixing errors, estimation in additive frameworks, estimation of the order of the autoregression). In addition we show that under a weak moment condition on the errors, our estimator is adaptive in the minimax sense simultaneously over some family of Besov balls.
MSC:
| 62G08 | Nonparametric regression and quantile regression |
| 62J02 | General nonlinear regression |
| 62M10 | Time series, auto-correlation, regression, etc. in statistics (GARCH) |
References:
| [1] | Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Proceedings of the 2nd International Symposium on Information Theory (P. N. Petrov and F. Csaki, eds.) 267-281. Akademia Kiado, Budapest. · Zbl 0283.62006 |
| [2] | Akaike, H. (1984). A new look at the statistical model identification. IEEE Trans. Automatic Control 19 716-723. · Zbl 0314.62039 · doi:10.1109/TAC.1974.1100705 |
| [3] | Baraud, Y. (1998). Sélection de mod eles et estimation adaptative dans différents cadres de régression. Ph.D. thesis, Univ. Paris-Sud. |
| [4] | Baraud, Y. (2000). Model selection for regression on a fixed design. Probab. Theory Related Fields 117 467-493. · Zbl 0997.62027 · doi:10.1007/s004400000058 |
| [5] | Baraud, Y. (2001). Model selection for regression on a random design. Preprint 01-10, DMA, Ecole Normale Supérieure, Paris. · Zbl 1059.62038 |
| [6] | Barron, A. R. (1991). Complexity regularization with application to artificial neural networks. In Proceedings of the NATO Advanced Study Institute on Nonparametric Functional Estimation (G. Roussas, ed.) 561-576. Kluwer, Dordrecht. · Zbl 0739.62001 |
| [7] | Barron, A. R. (1993). Universal approximation bounds for superpositions of a sigmoidal function processes. IEEE Trans. Inform. Theory 39 930-945. · Zbl 0818.68126 · doi:10.1109/18.256500 |
| [8] | Barron, A., Birgé, L. and Massart, P. (1999). Risks bounds for model selection via penalization. Probab. Theory Related Fields 113 301-413. · Zbl 0946.62036 · doi:10.1007/s004400050210 |
| [9] | Barron, A. R. and Cover, T. M. (1991). Minimum complexity density estimation. IEEE Trans. Inform. Theory 37 1034-1054. · Zbl 0743.62003 · doi:10.1109/18.86996 |
| [10] | Berbee, H. C. P. (1979). Random walks with stationary increments and renewal theory. Math. Centre Tract 112. Math. Centrum, Amsterdam. · Zbl 0443.60083 |
| [11] | Birgé, L. and Massart, P. (1997). From model selection to adaptive estimation. In Festschrift for Lucien Lecam: Research Papers in Probability and Statistics (D. Pollard, E. Torgensen and G. Yangs, eds.) 55-87. Springer, New York. · Zbl 0920.62042 |
| [12] | Birgé, L. and Massart, P. (1998). Exponential bounds for minimum contrast estimators on sieves. Bernoulli 4 329-375. · Zbl 0954.62033 |
| [13] | Cohen, A., Daubechies, I. and Vial, P. (1993). Wavelet and fast wavelet transform on an interval. Appl. Comp. Harmon. Anal. 1 54-81. · Zbl 0795.42018 · doi:10.1006/acha.1993.1005 |
| [14] | Daubechies, I. (1992). Ten Lectures on Wavelets. SIAM, Philadelphia. · Zbl 0776.42018 |
| [15] | Devore, R. A. and Lorentz, C. G. (1993). Constructive Approximation. Springer, New York. · Zbl 0797.41016 |
| [16] | Donoho, D. L. and Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist. 26 879-921. · Zbl 0935.62041 · doi:10.1214/aos/1024691081 |
| [17] | Doukhan, P. (1994). Mixing properties and Examples. Springer, New York. · Zbl 0790.60037 |
| [18] | Doukhan, P., Massart, P. and Rio, E. (1995). Invariance principle for absolutely regular empirical processes. Ann. Inst. H. Poincaré Probab. Statist. 31 393-427. · Zbl 0817.60028 |
| [19] | Duflo, M. (1997). Random Iterative Models. Springer, New-York. · Zbl 0868.62069 |
| [20] | Giné, E. and Zinn, J. (1984). Some limit theorems for empirical processes. Ann. Probab. 12 929- 989. · Zbl 0553.60037 · doi:10.1214/aop/1176993138 |
| [21] | Hoffmann, M. (1999). On nonparametric estimation in nonlinear AR(1)-models. Statist. Probab. Lett. 44 29-45. · Zbl 0954.62049 · doi:10.1016/S0167-7152(98)00289-2 |
| [22] | Kolmogorov, A. R. and Rozanov, Y. A. (1960). On the strongmixingconditions for stationary gaussian sequences. Theor. Probab. Appl. 5 204-207. · Zbl 0106.12005 · doi:10.1137/1105018 |
| [23] | Ledoux, M. and Talagrand, M. (1991). Probability in Banach Spaces. Springer, New York. · Zbl 0748.60004 |
| [24] | Li, K. C. (1987). Asymptotic optimality for Cp, Cl cross-validation and genralized cross-validation: discrete index set. Ann. Statist. 15 958-975. · Zbl 0653.62037 · doi:10.1214/aos/1176350486 |
| [25] | Mallows, C. L. (1973). Some comments on Cp. Technometrics 15 661-675. Modha, D. S. and Masry, E. (1996) Minimum complexity regression estimation with weakly dependent observations. IEEE Trans. Inform. Theory 42 2133-2145. · Zbl 0269.62061 |
| [26] | Modha, D. S. and Masry, E. (1998). Memory-universal prediction of stationary random processes. IEEE Trans. Inform. Theory 44 117-133. · Zbl 0938.62106 · doi:10.1109/18.650998 |
| [27] | Neumann, M. and Kreiss, J.-P. (1998). Regression-type inference in nonparametric autoregression. Ann. Statist. 26 1570-1613. · Zbl 0935.62049 · doi:10.1214/aos/1024691254 |
| [28] | Pham, D. T. and Tran, L. T. (1985). Some mixingproperties of time series models. Stochastic Process. Appl. 19 297-303. · Zbl 0564.62068 · doi:10.1016/0304-4149(85)90031-6 |
| [29] | Polyak, B. T. and Tsybakov, A. (1992). A family of asymptotically optimal methods for choosing the order of a projective regression estimate. Theory Probab. Appl. 37 471-481. · Zbl 0806.62031 · doi:10.1137/1137097 |
| [30] | Rissanen, J. (1984). Universal coding, information, prediction and estimation. IEEE Trans. Inform. Theory 30 629-636. · Zbl 0574.62003 · doi:10.1109/TIT.1984.1056936 |
| [31] | Shibata, R. (1976). Selection of the order of an autoregressive model by Akaike’s information criterion. Biometrika 63 117-126. JSTOR: · Zbl 0358.62048 · doi:10.1093/biomet/63.1.117 |
| [32] | Shibata, R. (1981). An optimal selection of regression variables. Biometrika 68 45-54. JSTOR: · Zbl 0464.62054 · doi:10.1093/biomet/68.1.45 |
| [33] | Talagrand, M. (1996). New concentration inequalities in product spaces. Invent. Math. 126 505- 563. · Zbl 0893.60001 · doi:10.1007/s002220050108 |
| [34] | Viennet, G. (1997). Inequalities for absolutely regular processes: application to density estimation. Probab. Theory Related Fields 107 467-492. · Zbl 0933.62029 · doi:10.1007/s004400050094 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
