Non-parametric sequential estimation of a regression function based on dependent observations. (English) Zbl 1294.62065
Summary: This article presents a sequential estimation procedure for an unknown regression function. Observed regressors and noises of the model are supposed to be dependent and form sequences of dependent numbers. Two types of estimators are considered. Both estimators are constructed on the basis of Nadaraya-Watson kernel estimators.
First, sequential estimators with given bias and mean square error are defined. According to the sequential approach the duration of observations is a special stopping time. Then on the basis of these estimators of a regression function, truncated sequential estimators on a time interval of a fixed length are constructed. At the same time, the variance of these estimators is controlled by a (non-asymptotic) bound.
In addition to nonasymptotic properties, the limiting behavior of presented estimators is investigated. It is shown, in particular, that by the appropriate chosen bandwidths both estimators have optimal (as compared to the case of independent data) rates of convergence of Nadaraya-Watson kernel estimators.
First, sequential estimators with given bias and mean square error are defined. According to the sequential approach the duration of observations is a special stopping time. Then on the basis of these estimators of a regression function, truncated sequential estimators on a time interval of a fixed length are constructed. At the same time, the variance of these estimators is controlled by a (non-asymptotic) bound.
In addition to nonasymptotic properties, the limiting behavior of presented estimators is investigated. It is shown, in particular, that by the appropriate chosen bandwidths both estimators have optimal (as compared to the case of independent data) rates of convergence of Nadaraya-Watson kernel estimators.
MSC:
| 62G05 | Nonparametric estimation |
| 62G08 | Nonparametric regression and quantile regression |
| 62G20 | Asymptotic properties of nonparametric inference |
| 62J20 | Diagnostics, and linear inference and regression |
| 62L12 | Sequential estimation |
Keywords:
dependent observations; finite sample size; guaranteed mean square accuracy; nonparametric kernel regression estimation; sequential approachReferences:
| [1] | DOI: 10.1109/TAC.2007.900826 · Zbl 1366.93667 · doi:10.1109/TAC.2007.900826 |
| [2] | Borisov V. Z., Automation and Remote Control 10 pp 58– (1977) |
| [3] | Carroll R. J., Zeitschrift für Warscheinlich- skeitstheorie und Verwandte Gebiete 36 pp 58– (1976) |
| [4] | DOI: 10.1109/TIT.1975.1055468 · Zbl 0318.62071 · doi:10.1109/TIT.1975.1055468 |
| [5] | Dedecker J., Stochastic Processes and Their Applications 10 pp 63– (1990) |
| [6] | Dharmasena S., Australian and New Zealand Industrial and Applied Mathematics Journal 49 pp 681– (2008) |
| [7] | Dobrovidov A. V., Non-Parametric State Space Models (2012) · Zbl 1284.62028 |
| [8] | Efroimovich S., Nonparametric Curve Estimation. Methods, Theory and Applications (1999) |
| [9] | DOI: 10.1080/07474940601109670 · Zbl 1112.62080 · doi:10.1080/07474940601109670 |
| [10] | DOI: 10.3103/S1066530709010037 · Zbl 1282.62042 · doi:10.3103/S1066530709010037 |
| [11] | DOI: 10.1007/b97848 · Zbl 1021.62024 · doi:10.1007/b97848 |
| [12] | DOI: 10.1080/07474949808836411 · Zbl 0914.62058 · doi:10.1080/07474949808836411 |
| [13] | Konev V. V., Sequential Parameter Estimation of Stochastic Dynamical Systems (1985) · Zbl 0582.93060 |
| [14] | DOI: 10.1080/07474949008836205 · Zbl 0704.62064 · doi:10.1080/07474949008836205 |
| [15] | DOI: 10.1080/07474949008836194 · Zbl 0698.62080 · doi:10.1080/07474949008836194 |
| [16] | Konev V. V., Methods of Economical Analysis pp 3– (1992) |
| [17] | DOI: 10.1016/j.automatica.2010.01.003 · Zbl 1193.93167 · doi:10.1016/j.automatica.2010.01.003 |
| [18] | DOI: 10.1016/S0167-7152(98)00256-9 · Zbl 0951.62035 · doi:10.1016/S0167-7152(98)00256-9 |
| [19] | DOI: 10.1007/978-1-4757-1665-8 · doi:10.1007/978-1-4757-1665-8 |
| [20] | DOI: 10.1007/978-1-4757-4293-0 · doi:10.1007/978-1-4757-4293-0 |
| [21] | Liptser R., Theory of Martingales (1988) |
| [22] | DOI: 10.1007/978-1-4613-0049-6 · doi:10.1007/978-1-4613-0049-6 |
| [23] | Loeve M., Probability Theory, (1963) |
| [24] | DOI: 10.1214/aos/1176348657 · Zbl 0746.62041 · doi:10.1214/aos/1176348657 |
| [25] | DOI: 10.1080/10485250310001622596 · Zbl 1072.62029 · doi:10.1080/10485250310001622596 |
| [26] | Mukhopadhyay N., Multistage Selection and Ranking Procedures (1994) · Zbl 0834.62072 |
| [27] | Nazin , A. V. and Ljung , L. ( 2002 ). Asymptotically Optimal Smoothing of Averaged LMS for Regression, Paper presented at 15th IFAC World Congress, Barcelona, Spain, July 21–26, 15: 436. |
| [28] | Novikov A. A., Theory of Probability and Its Applications 16 pp 394– (1971) |
| [29] | Politis D. N., Probability and Statistical Models with Applications pp 469– (2001) |
| [30] | Politis , D. N. and Romano , J. P. ( 1993 ). On a Family of Smoothing Kernels of Infinite Order, inProceedings of 25th Symposium on the Interface, San Diego, California, April 14–17, M. Tarter and M. Lock, eds., pp. 141–145, The Interface Foundation of North America. |
| [31] | DOI: 10.1006/jmva.1998.1774 · Zbl 0954.62042 · doi:10.1006/jmva.1998.1774 |
| [32] | DOI: 10.1007/978-1-4612-1554-7 · doi:10.1007/978-1-4612-1554-7 |
| [33] | Politis , D. N. and Vasiliev , V. A. ( 2012 ). Sequential Kernel Estimation of a Multivariate Regression Function, inProceedings of IX International Conference ’System Identification and Control Problems’, SICPRO’12, Moscow, 30 January–2 February, 2012. S. N. Vassiliev et al. eds., pp. 996–1009, Moscow: V. A. Tapeznikov Institute of Control Sciences. |
| [34] | Prakasa Rao B. L. S., Nonparametric Functional Estimation (1983) · Zbl 0542.62025 |
| [35] | Roll , J. , Nazin , A. , and Ljung , L. ( 2002 ). A Non-Asymptotic Approach to Local Modelling, the 41st IEEE CDC, Las Vegas, NV, December 10–13. Available at http://www.control.isy.liu.se/research/reports/2003/2482.pdf |
| [36] | DOI: 10.1016/j.automatica.2004.11.010 · Zbl 1127.93060 · doi:10.1016/j.automatica.2004.11.010 |
| [37] | Rosenblatt M., Stochastic Curve Estimation, Vol. 3, NSF-CBMS Regional Conference Series (1991) · Zbl 1163.62318 |
| [38] | DOI: 10.1002/9780470316849 · doi:10.1002/9780470316849 |
| [39] | DOI: 10.1007/BF00532165 · Zbl 0488.62032 · doi:10.1007/BF00532165 |
| [40] | Vasiliev V. A., Automation and Remote Control 58 pp 106– (1997) |
| [41] | Vasiliev V. A., Preprints of 18th IFAC World Congress (2011) |
| [42] | Vasiliev , V. A. and Konev , V. V. ( 1997 ). On Optimal Adaptive Control of a Linear Stochastic Process,Proceedings of 15th IMACS World Congress on Scientific Computation, Modelling and Applied Mathematics, August 24–29, Berlin, Germany 5: 87–91. |
| [43] | Vasiliev V. A., Probability Theory and Its Applications 43 pp 577– (1998) |
| [44] | Wald A., Sequential Analysis (1947) · Zbl 0029.15805 |
| [45] | Wald A., Statistical Decision Functions (1950) · Zbl 0040.36402 |
| [46] | DOI: 10.1214/aos/1176324621 · Zbl 0842.62005 · doi:10.1214/aos/1176324621 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
