A consistent test for nonlinear out of sample predictive accuracy. (English) Zbl 1044.62118
Summary: We draw on both the consistent specification testing and the predictive ability testing literature and propose an integrated conditional moment type predictive accuracy test that is similar in spirit to that developed by H. J. Bierens [J. Econometr. 20, 105–134 (1982; Zbl 0549.62076); Econometrica 58, 1443–1458 (1990; Zbl 0737.62058)] and H. J. Bierens and W. Ploberger [ibid. 65, 1129–1157 (1997; Zbl 0927.62085)]. The test is consistent against generic nonlinear alternatives, and is designed for comparing nested models. One important feature of our approach is that the same loss function is used for in-sample estimation and out-of-sample prediction. In this way, we rule out the possibility that the null model can outperform the nesting generic alternative model.
It turns out that the limiting distribution of the ICM type test statistic that we propose is a functional of a Gaussian process with a covariance kernel that reflects both the time series structure of the data as well as the contribution of parameter estimation error. As a consequence, critical values are data dependent and cannot be directly tabulated. One approach in this case is to obtain critical value upper bounds using the approach of Bierens and Ploberger (loc. cit.). Here, we establish the validity of a conditional \(p\)-value method for constructing critical values. The method is similar in spirit to that proposed by B. E. Hansen [ibid. 64, 413–430 (1996; Zbl 0862.62090)] and A. Inoue [Econom. Theory 17, 156–187 (2001; Zbl 0976.62088)], although we additionally account for parameter estimation error.
In a series of Monte Carlo experiments, the finite sample properties of three variants of the predictive accuracy test are examined. Our findings suggest that all three variants of the test have good finite sample properties when quadratic loss is specified, even for samples as small as 600 observations. However, non-quadratic loss functions, such as linex loss, require larger sample sizes (of 1000 observations or more) in order to ensure reasonable finite sample performance.
It turns out that the limiting distribution of the ICM type test statistic that we propose is a functional of a Gaussian process with a covariance kernel that reflects both the time series structure of the data as well as the contribution of parameter estimation error. As a consequence, critical values are data dependent and cannot be directly tabulated. One approach in this case is to obtain critical value upper bounds using the approach of Bierens and Ploberger (loc. cit.). Here, we establish the validity of a conditional \(p\)-value method for constructing critical values. The method is similar in spirit to that proposed by B. E. Hansen [ibid. 64, 413–430 (1996; Zbl 0862.62090)] and A. Inoue [Econom. Theory 17, 156–187 (2001; Zbl 0976.62088)], although we additionally account for parameter estimation error.
In a series of Monte Carlo experiments, the finite sample properties of three variants of the predictive accuracy test are examined. Our findings suggest that all three variants of the test have good finite sample properties when quadratic loss is specified, even for samples as small as 600 observations. However, non-quadratic loss functions, such as linex loss, require larger sample sizes (of 1000 observations or more) in order to ensure reasonable finite sample performance.
MSC:
| 62P20 | Applications of statistics to economics |
| 62E20 | Asymptotic distribution theory in statistics |
| 65C05 | Monte Carlo methods |
Keywords:
conditional \(p\)-value; nonlinear causality; out of sample predictive accuracy; parameter estimation error; tablesCitations:
Zbl 0737.62058; Zbl 0549.62076; Zbl 0927.62085; Zbl 0862.62090; Zbl 0976.65088; Zbl 0976.62088References:
| [1] | Altissimo, F., Corradi, V., 2001. Bounds for inference in the presence of nuisance parameters unidentified under the null. Mimeo, University of Exeter.; Altissimo, F., Corradi, V., 2001. Bounds for inference in the presence of nuisance parameters unidentified under the null. Mimeo, University of Exeter. · Zbl 1018.62104 |
| [2] | Andrews, D. W.K.; Ploberger, W., Optimal tests when a nuisance parameter is present only under the alternative, Econometrica, 62, 1383-1414 (1994) · Zbl 0815.62033 |
| [3] | Bierens, H. B., Consistent model specification tests, Journal of Econometrics, 20, 105-134 (1982) · Zbl 0549.62076 |
| [4] | Bierens, H. B., A conditional moment test of functional form, Econometrica, 58, 1443-1458 (1990) · Zbl 0737.62058 |
| [5] | Bierens, H. J.; Ploberger, W., Asymptotic theory of integrated conditional moment tests, Econometrica, 65, 1129-1152 (1997) · Zbl 0927.62085 |
| [6] | Chao, J. C.; Corradi, V.; Swanson, N. R., An out of sample test for granger causality, Macroeconomic Dynamics, 5, 598-620 (2001) · Zbl 1005.91083 |
| [7] | Christoffersen, P.; Diebold, F. X., Further results on forecasting and model selection under asymmetric loss, Journal of Applied Econometrics, 11, 561-572 (1996) |
| [8] | Christoffersen, P.; Diebold, F. X., Optimal prediction under asymmetric loss, Econometric Theory, 13, 808-817 (1997) |
| [9] | Clark, T. E.; McCracken, M. W., Tests of equal forecast accuracy and encompassing for nested models, Journal of Econometrics, 105, 85-110 (2001) · Zbl 0980.62105 |
| [10] | Clements, M. P.; Hendry, D. F., Forecasting Economic Time Series: the Zeuthen Lectures on Economic Forecasting (1999), MIT Press: MIT Press Cambridge · Zbl 0984.62069 |
| [11] | Clements, M. P.; Hendry, D. F., On winning forecasting competitions in economics, Spanish Economic Review, 1, 123-160 (1999) |
| [12] | Clements, M.P., Hendry, D.F., 2001. Forecasting with difference and trend stationary models. The Econometrics Journal, Forthcoming.; Clements, M.P., Hendry, D.F., 2001. Forecasting with difference and trend stationary models. The Econometrics Journal, Forthcoming. · Zbl 1004.62074 |
| [13] | Corradi, V., Swanson, N.R., 2001. Bootstrapping conditional distributions tests under dynamic misspecification. Mimeo, University of Exeter.; Corradi, V., Swanson, N.R., 2001. Bootstrapping conditional distributions tests under dynamic misspecification. Mimeo, University of Exeter. |
| [14] | Corradi, V.; Swanson, N. R.; Olivetti, C., Predictive ability with cointegrated variables, Journal of Econometrics, 104, 315-358 (2001) · Zbl 1066.62544 |
| [15] | de Jong, R. M., The Bierens test under data dependence, Journal of Econometrics, 72, 1-32 (1996) · Zbl 0855.62073 |
| [16] | de Jong, R. M., A strong consistency proof for heteroskedasticity and autocorrelation consistent covariance estimators, Econometric Theory, 16, 262-269 (2000) · Zbl 0957.62074 |
| [17] | Diebold, F. X.; Mariano, R. S., Comparing predictive accuracy, Journal of Business and Economic Statistics, 13, 253-263 (1995) |
| [18] | Doukhan, P., 1995. Mixing: Properties and Examples. Lecture Notes in Statistics, Springer, Berlin.; Doukhan, P., 1995. Mixing: Properties and Examples. Lecture Notes in Statistics, Springer, Berlin. · Zbl 0801.60027 |
| [19] | Eberlain, E., On strong invariance principles under dependence assumptions, Annals of Probability, 14, 260-270 (1986) · Zbl 0589.60031 |
| [20] | Granger, C. W.J., Investigating causal relations by econometric models and cross spectral methods, Econometrica, 37, 424-438 (1969) · Zbl 1366.91115 |
| [21] | Granger, C. W.J., Testing for causality: a personal viewpoint, Journal of Economic Dynamics and Control, 2, 329-352 (1980) |
| [22] | Granger, C. W.J., On the limitations of comparing mean squared forecast errors: comment, Journal of Forecasting, 12, 651-652 (1993) |
| [23] | Granger, C. W.J., Outline of forecast theory using generalized cost functions, Spanish Economic Review, 1, 161-173 (1999) |
| [24] | Granger, C. W.J.; Newbold, P., Forecasting Economic Time Series (1986), Academic Press: Academic Press San Diego · Zbl 0642.90001 |
| [25] | Granger, C. W.J.; Teräsvirta, T., Modelling Nonlinear Economic Relationships (1993), Oxford University Press: Oxford University Press Oxford · Zbl 0893.90030 |
| [26] | Hansen, B. E., Inference when a nuisance parameter is not identified under the null hypothesis, Econometrica, 64, 413-430 (1996) · Zbl 0862.62090 |
| [27] | Harvey, D. I.; Leybourne, S. J.; Newbold, P., Tests for forecast encompassing, Journal of Business and Economic Statistics, 16, 254-259 (1997) |
| [28] | Inoue, A., Testing for distributional change in time series, Econometric Theory, 17, 156-187 (2001) · Zbl 0976.62088 |
| [29] | Künsch, H. R., The jackknife and the bootstrap for general stationary observations, Annals of Statistics, 17, 1217-1241 (1989) · Zbl 0684.62035 |
| [30] | Lee, T.-H.; White, H.; Granger, C. W.J., Testing for neglected nonlinearity in time series models: a comparison of neural network methods and alternative tests, Journal of Econometrics, 56, 269-290 (1993) · Zbl 0766.62055 |
| [31] | Luukkonen, R.; Saikkonen, P.; Teräsvirta, T., Testing linearity against smooth transition autoregressive models, Biometrika, 75, 491-499 (1988) · Zbl 0657.62109 |
| [32] | McCracken, M.W., 1999. Asymptotics for out of sample tests of causality. Working Paper, Louisiana State University.; McCracken, M.W., 1999. Asymptotics for out of sample tests of causality. Working Paper, Louisiana State University. · Zbl 1247.91150 |
| [33] | McCracken, M. W., Robust out of sample inference, Journal of Econometrics, 99, 195-223 (2000) · Zbl 0964.62093 |
| [34] | Politis, D.; Romano, J., The stationary bootstrap, Journal of the American Statistical Association, 89, 1303-1313 (1994) · Zbl 0814.62023 |
| [35] | Pollard, D., 1990. Empirical Processes: Theory and Applications, Vol. 2. CBMS, Conference Series in Probability and Statistics. Hayward, CA, Institute of Mathematical Statistics.; Pollard, D., 1990. Empirical Processes: Theory and Applications, Vol. 2. CBMS, Conference Series in Probability and Statistics. Hayward, CA, Institute of Mathematical Statistics. · Zbl 0741.60001 |
| [36] | Ramsey, J. B., Tests for specification errors in classical linear least squares regression analysis, Journal of the Royal Statistical Society, Series B, 31, 350-371 (1969) · Zbl 0179.48902 |
| [37] | Rothman, P.; van Dijk, D. J.C.; Franses, P. H., A multivariate STAR analysis of the relationship between money and output, Macroeconomic Dynamics, 5, 506-532 (2001) · Zbl 1001.91076 |
| [38] | Stinchcombe, M. B.; White, H., Consistent specification testing with nuisance parameters present only under the alternative, Econometric Theory, 14, 3, 295-325 (1998) · Zbl 1419.62105 |
| [39] | Sullivan, R.; Timmermann, A.; White, H., Dangers of data-driven inference: the case of calendar effects in stock returns, Journal of Finance, 54, 1647-1691 (1999) |
| [40] | Swanson, N. R.; White, H., A model selection approach to assessing the information in the term structure using linear models and artificial neural networks, Journal of Business and Economic and Statistics, 13, 265-275 (1995) |
| [41] | Swanson, N. R.; White, H., A model selection approach to real-time macroeconomic forecasting using linear models and artificial neural networks, Review of Economics and Statistics, 79, 540-550 (1997) |
| [42] | Teräsvirta, T.; Anderson, H., Characterizing nonlinearities in business cycles using smooth transition autoregressive models, Review of Journal of Applied Econometrics, 7, 119-136 (1992) |
| [43] | Weiss, A., Estimating time series models using the relevant cost function, Journal of Applied Econometrics, 11, 539-560 (1996) |
| [44] | West, K., Asymptotic inference about predictive ability, Econometrica, 64, 1067-1084 (1996) · Zbl 0854.62101 |
| [45] | West, K.; McCracken, M. W., Regression based tests of predictive ability, International Economic Review, 39, 817-840 (1998) |
| [46] | White, H., Estimation, Inference and Specification Analysis (1994), Cambridge University Press: Cambridge University Press Cambridge · Zbl 0860.62100 |
| [47] | White, H., A reality check for data snooping, Econometrica, 68, 1097-1126 (2000) · Zbl 1008.62116 |
| [48] | Wolkonski, V. A.; Rozanov, Y. A., Some limit theorems for random functions, Part I, Theory of Probability and Its Applications, 6, 186-198 (1959) |
| [49] | Zellner, A., Bayesian estimation and prediction using asymmetric loss function, Journal of the American Statistical Association, 81, 446-451 (1986) · Zbl 0603.62037 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
