Statistical tests for multiple forecast comparison. (English) Zbl 1443.62491
Summary: We consider a multivariate version of the Diebold-Mariano test for equal predictive ability of three or more forecasting models. The Wald-type test, \(S\), which has a null distribution that is asymptotically chi-squared, is shown to be generally invariant with respect to the ordering of the models being compared. Finite-sample corrections for the test are also developed. Monte Carlo simulations indicate that \(S\) has reasonable size properties in large samples but tends to be oversized in moderate samples. The finite-sample correction succeeds in correcting for size, but only partially. For the size-adjusted tests, power increases with sample size, as expected. It is speculated that further finite-sample improvements can be achieved using Hotelling’s \(T^2\) or bootstrap critical values.
MSC:
| 62P20 | Applications of statistics to economics |
| 62H15 | Hypothesis testing in multivariate analysis |
| 62M20 | Inference from stochastic processes and prediction |
Keywords:
forecast comparison; multivariate tests of equal predictive ability; Diebold-Mariano test; finite-sample correctionReferences:
| [1] | Andrews, D. W.K., Heteroskedasticity and autocorrelation consistent covariance matrix estimation, Econometrica, 59, 817-858 (1991) · Zbl 0732.62052 |
| [2] | Christiano, L. J., \(p^\ast \): not the inflation forecasters holy grail, Federal Reserve Bank of Minneapolis Quarterly Review, 13, 3-18 (1989) |
| [3] | Clark, T. E.; West, K. D., Using out-of-sample mean squared prediction errors to test the martingale difference hypothesis, Journal of Econometrics, 135, 155-186 (2006) · Zbl 1418.62433 |
| [4] | Clark, T. E.; West, K. D., Approximately normal tests for equal predictive accuracy in nested models, Journal of Econometrics, 138, 291-311 (2007) · Zbl 1418.62432 |
| [5] | Corradi, V.; Swanson, N. R., Predictive density and conditional confidence interval accuracy tests, Journal of Econometrics, 135, 187-228 (2006) · Zbl 1418.62438 |
| [6] | Corradi, V.; Swanson, N. R., Nonparametric bootstrap procedures for predictive inference based on recursive estimation schemes, International Economic Review, 48, 67-109 (2007) |
| [7] | Corradi, V.; Swanson, N. R.; Olivetti, C., Predictive ability with cointegrated variables, Journal of Econometrics, 104, 315-358 (2001) · Zbl 1066.62544 |
| [8] | Diebold, F. X.; Mariano, R. S., Comparing predictive accuracy, Journal of Business & Economic Statistics, 13, 134-145 (1995) |
| [9] | Giacomini, R.; White, H., Tests of conditional predictive ability, Econometrica, 74, 1545-1578 (2006) · Zbl 1187.91151 |
| [10] | Granger, C.; Newbold, P., Forecasting Economic Time Series (1977), Academic Press |
| [11] | Hansen, P. R., A test for superior predictive ability, Journal of Business & Economic Statistics, 23, 365-380 (2005) |
| [12] | Hansen, P. R.; Lunde, A.; Nason, J. M., The model confidence set, Econometrica, 79, 453-497 (2011) · Zbl 1210.62030 |
| [13] | Harvey, D.; Leybourne, S.; Newbold, P., Testing the equality of prediction mean squared errors, International Journal of Forecasting, 13, 281-291 (1997) |
| [14] | Hubrich, K.; West, K. D., Forecast evaluation of small nested model sets, Journal of Applied Econometrics, 25, 574-594 (2010) |
| [15] | Lawrence, M.; Goodwin, P.; O’Connor, M.; Önkal, D., Judgmental forecasting: a review of progress over the last 25 years, International Journal of Forecasting, 22, 493-518 (2006) |
| [16] | Mardia, K.; Kent, J.; Bibby, J., Multivariate Analysis (2000), Academic Press · Zbl 0432.62029 |
| [17] | Mariano, R. S., Testing forecast accuracy, (Clements, M. P.; Hendry, D. F., A Companion to Economic Forecasting (2002), Wiley-Blackwell), 284-298 · Zbl 1092.62053 |
| [18] | McCracken, M. W., Robust out-of-sample inference, Journal of Econometrics, 99, 195-223 (2000) · Zbl 0964.62093 |
| [19] | McCracken, M. W., Asymptotics for out of sample tests of Granger causality, Journal of Econometrics, 140, 719-752 (2007) · Zbl 1247.91150 |
| [20] | Meese, R.; Rogoff, K., Was it real? The exchange rate-interest rate differential relation over the modern floating-rate period, Journal of Finance, 43, 933-948 (1988) |
| [21] | Morgan, W. A., A test for significance of the difference between the two variances in a sample from a normal bivariate population, Biometrika, 31, 13-19 (1939) · Zbl 0022.25001 |
| [22] | Newey, W. K.; West, K. D., A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix, Econometrica, 55, 703-708 (1987) · Zbl 0658.62139 |
| [23] | Romano, J. P.; Wolf, M., Stepwise multiple testing as formalized data snooping, Econometrica, 73, 1237-1282 (2005) · Zbl 1153.62310 |
| [24] | Tiao, G. C.; Box, G. E.P., Modeling multiple time series with applications, Journal of the American Statistical Association, 76, 802-816 (1981) · Zbl 0483.62074 |
| [25] | West, K. D., Asymptotic inference about predictive ability, Econometrica, 64, 1067-1084 (1996) · Zbl 0854.62101 |
| [26] | West, K. D., Heteroskedasticity and autocorrelation corrections, (Durlauf, S. N.; Blume, L. E., The New Palgrave Dictionary of Economics (2008), Palgrave Macmillan), 6-12 |
| [27] | West, K. D.; Cho, D., The predictive ability of several models of exchange rate volatility, Journal of Econometrics, 69, 367-391 (1995) |
| [28] | West, K. D.; Edison, H. J.; Cho, D., A utility-based comparison of some models of exchange rate volatility, Journal of International Economics, 35, 23-45 (1993) |
| [29] | White, H., A reality check for data snooping, Econometrica, 68, 1097-1126 (2000) · Zbl 1008.62116 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
