×

Estimating the Kullback-Leibler risk based on multifold cross-validation. (Estimating the Kullback-Liebler risk based on multifold cross-validation.) (English) Zbl 1541.62077

Summary: This paper concerns a class of model selection criteria based on cross-validation techniques and estimative predictive densities. Both the simple or leave-one-out and the multifold or leave-\(m\)-out cross-validation procedures are considered. These cross-validation criteria define suitable estimators for the expected Kullback-Liebler risk, which measures the expected discrepancy between the fitted candidate model and the true one. In particular, we shall investigate the potential bias of these estimators, under alternative asymptotic regimes for \(m\). The results are obtained within the general context of independent, but not necessarily identically distributed, observations and by assuming that the candidate model may not contain the true distribution. An application to the class of normal regression models is also presented, and simulation results are obtained in order to gain some further understanding on the behavior of the estimators.
{© 2015 The Author. Statistica Neerlandica © 2015 VVS.}

MSC:

62G05 Nonparametric estimation
62B10 Statistical aspects of information-theoretic topics

References:

[1] AllenD. M. (1974), The relationship between variable selection and data augmentation and a method for prediction, Technometrics16, 125-127. · Zbl 0286.62044
[2] AkaikeH. (1973), Information theory and extension of the maximum likelihood principle, in: N. B.Petron (ed.) and F.Caski (ed.) (eds), Second symposium on information theory, Budapest, Akademiai Kiado, 267-281. · Zbl 0283.62006
[3] ArlotS. (2008), V‐fold cross‐validation improved: V‐fold penalization,. arXiv: 0802.0566v2.
[4] ArlotS. and A.Celisse (2010), A survey of cross‐validation procedures for model selection, Statistics Surveys4, 40-79. · Zbl 1190.62080
[5] Barndorff‐NielsenO. E. and D. R.Cox (1994), Inference and asymptotics, Chapman and Hall, London. · Zbl 0826.62004
[6] BurmanP. (1989), A comparative study of ordinary cross‐validation, υ‐fold cross‐validation and the repeated learning‐testing methods, Biometrika76, 503-514. · Zbl 0677.62065
[7] BurnhamK. P. and D. R.Anderson (2002), Model selection and multimodel inference 2nd edition, Springer‐Verlag, New York. · Zbl 1005.62007
[8] CelisseA. (2014), Optimal cross‐validation in density estimation with the L^2‐loss, Annals of Statistics42, 1879-1910. · Zbl 1305.62179
[9] DavisonA. C. and D. V.Hinkley (1997), Bootstrap methods and their application, Cambridge University Press, Cambridge. · Zbl 0886.62001
[10] DraperN. R. and H.Smith (1998), Applied regression analysis 3rd edition, John Wiley, New York. · Zbl 0895.62073
[11] EfronB. (1983), Estimating the error rate of a prediction rule: improvement of cross‐validation, Journal of the American Statistical Association78, 316-331. · Zbl 0543.62079
[12] EfronB. (1986), How biased is the apparent error rate of a prediction rule?Journal of the American Statistical Association81, 461-470. · Zbl 0621.62073
[13] FujikoshiY., T.Noguchi, M.Ohtaki and H.Yanagihara (2003), Corrected versions of cross‐validation criteria for selecting multivariate regression and growth models, Annals of the Institute of Statistical Mathematics55, 537-553. · Zbl 1047.62051
[14] FushikiT. (2011), Estimation of prediction error by using K‐fold cross‐validation, Statistics and Computing21, 137-146. · Zbl 1254.62099
[15] GeisserS. (1975), The predictive sample reuse method with applications, Journal of the American Statistical Association70, 320-328. · Zbl 0321.62077
[16] GeisserS. and W. F.Eddy (1979), A predictive approach to model selection, Journal of the American Statistical Association74, 153-160. · Zbl 0401.62036
[17] HerzbergG. and S.Tsukanov (1986), A note on modifications of the jackknife criterion on model selection, Utilitas Mathematica29, 209-216. · Zbl 0591.62063
[18] KonishiS. and G.Kitagawa (1996), Generalised information criteria in model selection, Biometrika83, 875-890. · Zbl 0883.62004
[19] LiK. C. (1987), Asymptotic optimality for C_p,C_L, cross‐validation and generalized cross‐validation: discrete index set, Annals of Statistics15, 958-975. · Zbl 0653.62037
[20] PicardR. R. and R. D.Cook (1984), Cross‐validation of regression models, Journal of the American Statistical Association79, 575-583. · Zbl 0547.62047
[21] ShaoJ. (1993), Linear model selection by cross‐validation, Journal of the American Statistical Association88, 486-494. · Zbl 0773.62051
[22] ShaoJ. (1997), An asymptotic theory for linear model selection, Statistica Sinica7, 221-264. · Zbl 1003.62527
[23] StoneM. (1974), Cross‐validation choice and assessment of statistical predictions, Journal of the Royal Statistical Society: Series B36, 111-147. · Zbl 0308.62063
[24] StoneM. (1977a), An asymptotic equivalence of choice of model by cross‐validation and Akaike’s criterion, Journal of the Royal Statistical Society: Series B39, 44-47. · Zbl 0355.62002
[25] StoneM. (1977b), Asymptotics for and against cross‐validation, Biometrika64, 29-38. · Zbl 0368.62046
[26] WhiteH. (1994), Estimation, inference, and specification analysis, Cambridge University Press, New York. · Zbl 0860.62100
[27] YanagiharaH. and H.Fujisawa (2012), Iterative bias correction of the cross‐validation criterion, Scandinavian Journal of Statistics39, 116-130. · Zbl 1246.62093
[28] YanagiharaH., T.Tonda and C.Matsumoto (2006), Bias correction of cross‐validation criterion based on Kullback-Leibler information under a general condition, Journal of Multivariate Analysis97, 1965-1975. · Zbl 1101.62047
[29] YangY. (2005), Can the strengths of AIC and BIC be shared? A conflict between model identification and regression estimation, Biometrika92, 937-950. · Zbl 1151.62301
[30] YangY. (2007), Consistency of cross validation for comparing regression procedures, Annals of Statistics35, 2450-2473. · Zbl 1129.62039
[31] ZhangP. (1993), Model selection via multifold cross‐validation, Annals of Statistics21, 299-313. · Zbl 0770.62053
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.