Regression model selection – a residual likelihood approach. (English) Zbl 1059.62074
Summary: We obtain the residual information criterion RIC, a selection criterion based on the residual log-likelihood, for regression models including classical regression models, Box-Cox transformation models, weighted regression models and regression models with autoregressive moving average errors. We show that RIC is a consistent criterion, and that simulation studies for each of the four models indicate that RIC provides better model order choices than the Akaike information criterion, corrected Akaike information criterion, final prediction error, \(C_p\) and \(R^2_{\text{adj}}\), except when the sample size is small and the signal-to-noise ratio is weak. In this case, none of the criteria performs well. Monte Carlo results also show that RIC is superior to the consistent Bayesian information criterion BIC when the signal-to-noise ratio is not weak, and it is comparable with BIC when the signal-to-noise ratio is weak and the sample size is large.
MSC:
| 62J05 | Linear regression; mixed models |
| 62B10 | Statistical aspects of information-theoretic topics |
| 65C05 | Monte Carlo methods |
Keywords:
Akaike information criterion; Bayesian information criterion; corrected Akaike information criterion; residual information criterion; residual likelihoodReferences:
| [1] | Akaike, Statistical predictor identification, Ann. Inst. Statist. Math. 22 pp 203– (1970) · Zbl 0259.62076 · doi:10.1007/BF02506337 |
| [2] | Proc. 2nd Int. Symp. Information Theory pp 267– (1973) |
| [3] | Bickel, An analysis of transformation revisited, J. Am. Statist. Ass. 76 pp 296– (1981) · doi:10.1080/01621459.1981.10477649 |
| [4] | Box, An analysis of transformations (with discussion), J. R. Statist. Soc. 26 pp 211– (1964) · Zbl 0156.40104 |
| [5] | Brown, Applied Mixed Models in Medicine (1999) |
| [6] | Burnham, Model Selection and Inference (a Practical Information-theoretic Approach) (1998) · Zbl 0920.62006 · doi:10.1007/978-1-4757-2917-7 |
| [7] | Carroll, Transformation and Weighting in Regression (1988) · doi:10.1007/978-1-4899-2873-3 |
| [8] | Chatterjee, Sensitivity Analysis in Linear Regression (1988) · doi:10.1002/9780470316764 |
| [9] | Cheang, Bias reduction of autoregressive estimates in time series regression model through restricted maximum likelihood, J. Am. Statist. Ass. 95 pp 1173– (2000) · Zbl 1004.62071 · doi:10.1080/01621459.2000.10474318 |
| [10] | Cooper, A note on the estimation of parameters of the autoregressive-moving average process, Biometrika 64 pp 625– (1977) · Zbl 0368.62076 |
| [11] | Corbeil, Restricted maximum likelihood (REML) estimation of variance components in the mixed model, Technometrics 18 pp 31– (1976) · Zbl 0324.62047 · doi:10.2307/1267913 |
| [12] | Diggle, Analysis of Longitudinal Data (1994) · Zbl 0825.62010 |
| [13] | Harvey, Maximum likelihood estimation of regression models with auto-regressive-moving average disturbances, Biometrika 66 pp 49– (1979) |
| [14] | Harville, Bayesian inference for variance components using only error contrasts, Biometrika 61 pp 383– (1974) · Zbl 0281.62072 · doi:10.1093/biomet/61.2.383 |
| [15] | He, Linear regression after spline transformation, Biometrika 84 pp 474– (1997) · Zbl 0883.62070 · doi:10.1093/biomet/84.2.474 |
| [16] | Hurvich, Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion, J. R. Statist. Soc. 60 pp 271– (1998) · Zbl 0909.62039 · doi:10.1111/1467-9868.00125 |
| [17] | Hurvich, Regression and time series model selection in small samples, Biometrika 76 pp 297– (1989) · Zbl 0669.62085 · doi:10.1093/biomet/76.2.297 |
| [18] | Jones, Longitudinal Data with Serial Correlation: a State-space Approach (1993) · Zbl 0851.62059 · doi:10.1007/978-1-4899-4489-4 |
| [19] | Lahiri, Joint estimation and testing for functional form and heteroscedasticity, J. Econometr. 15 pp 299– (1981) · Zbl 0449.62049 · doi:10.1016/0304-4076(81)90119-6 |
| [20] | Linhart, Model Selection (1986) |
| [21] | Lyon, A comparison of tests for heteroscedasticity, Statistician 45 pp 337– (1996) · doi:10.2307/2988471 |
| [22] | Mallows, Some comments on Cp, Technometrics 15 pp 661– (1973) · Zbl 0269.62061 |
| [23] | McCullagh, Generalized Linear Models (1989) · Zbl 0588.62104 · doi:10.1007/978-1-4899-3242-6 |
| [24] | McCulloch, Generalized, Linear, and Mixed Models (2000) · doi:10.1002/0471722073 |
| [25] | Patterson, Recovery of inter-block information when block sizes are unequal, Biometrika 8 pp 545– (1971) · Zbl 0228.62046 · doi:10.1093/biomet/58.3.545 |
| [26] | Rao, Estimation of Variance Components and Applications (1988) · Zbl 0645.62073 |
| [27] | Schwarz, Estimating the dimension of a model, Ann. Statist. 6 pp 461– (1978) · Zbl 0379.62005 · doi:10.1214/aos/1176344136 |
| [28] | Sen, Regression Analysis-Theory, Methods, and Applications (1990) · Zbl 0714.62057 |
| [29] | Simonoff, Semiparametric and additive model selection using an improved Akaike information criterion, J. Comput. Graph. Statist. 58 pp 22– (1999) |
| [30] | Tsay, Regression models with time series errors, J. Am. Statist. Ass. 79 pp 118– (1984) · Zbl 0533.62082 · doi:10.1080/01621459.1984.10477073 |
| [31] | Tunnicliffe Wilson, On the use of marginal likelihood in time series model estimation, J. R. Statist. Soc. 51 pp 15– (1989) |
| [32] | Verbyla, A conditional derivation of residual maximum likelihood, Aust. J. Statist. 32 pp 227– (1990) · doi:10.1111/j.1467-842X.1990.tb01015.x |
| [33] | Modelling variance heterogeneity: residual maximum likelihood and diagnostics, J. R. Statist. Soc. 55 pp 493– (1993) · Zbl 0783.62051 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
