Joint variable selection for fixed and random effects in linear mixed-effects models. (English) Zbl 1233.62134
Summary: It is of great practical interest to simultaneously identify the important predictors that correspond to both the fixed and random effects components in a linear mixed-effects (LME) model. Typical approaches perform selection separately on each of the fixed and random effect components. However, changing the structure of one set of effects can lead to different choices of variables for the other set of effects. We propose simultaneous selection of the fixed and random factors in an LME model using a modified Cholesky decomposition. Our method is based on a penalized joint log likelihood with an adaptive penalty for the selection and estimation of both the fixed and random effects. It performs model selection by allowing fixed effects or standard deviations of random effects to be exactly zero. A constrained expectation-maximization algorithm is then used to obtain the final estimates. It is further shown that the proposed penalized estimator enjoys the oracle property, in that asymptotically it performs as well as if the true model was known beforehand. We demonstrate the performance of our method based on a simulation study and a real data example.
MSC:
| 62J05 | Linear regression; mixed models |
| 62H12 | Estimation in multivariate analysis |
| 62P12 | Applications of statistics to environmental and related topics |
| 65C60 | Computational problems in statistics (MSC2010) |
Keywords:
adaptive LASSO; constrained EM algorithm; modified Cholesky decomposition; penalized likelihood; variable selection; air pollutionSoftware:
OSCARReferences:
| [1] | Akaike, Second International Symposium on Information Theory (1973) |
| [2] | Bondell, Simultaneous regression shrinkage, variable selection, and supervised clustering of predictors with OSCAR, Biometrics 64 pp 115– (2008) · Zbl 1146.62051 · doi:10.1111/j.1541-0420.2007.00843.x |
| [3] | Breiman, Heuristics of instability and stabilization in model selection, Annals of Statistics 24 pp 2350– (1996) · Zbl 0867.62055 · doi:10.1214/aos/1032181158 |
| [4] | Chen, Random effects selection in linear mixed models, Biometrics 59 pp 762– (2003) · Zbl 1214.62027 · doi:10.1111/j.0006-341X.2003.00089.x |
| [5] | Diggle, Analysis of Longitudinal Data (1994) · Zbl 0825.62010 |
| [6] | Efron, Least angle regression, Annals of Statistics 32 pp 407– (2004) · Zbl 1091.62054 · doi:10.1214/009053604000000067 |
| [7] | Fan, Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association 96 pp 1348– (2001) · Zbl 1073.62547 · doi:10.1198/016214501753382273 |
| [8] | Ghosh, Spatio-temporal analysis of total nitrate concentrations using dynamic statistical models, Journal of the American Statistical Association (2010) · Zbl 1392.62332 · doi:10.1198/jasa.2010.ap07441 |
| [9] | Hannan, The determination of the order of an autoregression, Journal of the Royal Statistical Society, Series B 41 pp 190– (1979) · Zbl 0408.62076 |
| [10] | Jiang, Consistent procedures for mixed linear model selection, Sankhya A 65 pp 23– (2003) · Zbl 1193.62112 |
| [11] | Jiang, Fence methods for mixed models selection, Annals of Statistics 36 pp 1669– (2008) · Zbl 1142.62047 · doi:10.1214/07-AOS517 |
| [12] | Kinney, Fixed and random effects selection in linear and logistic models, Biometrics 63 pp 690– (2007) · Zbl 1147.62022 · doi:10.1111/j.1541-0420.2007.00771.x |
| [13] | Kullback, On information and sufficiency, Annals of Mathematical Statistics 22 pp 72– (1951) · Zbl 0042.38403 · doi:10.1214/aoms/1177729694 |
| [14] | Laird, Random-effects models for longitudinal data, Biometrics 38 pp 963– (1982) · Zbl 0512.62107 · doi:10.2307/2529876 |
| [15] | Laird, Maximum likelihood computations with repeated measures: Application of the EM algorithm, Journal of the American Statistical Association 82 pp 97– (1987) · Zbl 0613.62063 · doi:10.2307/2289129 |
| [16] | Lange, The effect of covariance structures on variance estimation in balance growth-curve models with random parameters, Journal of the American Statistical Association 84 pp 241– (1989) · Zbl 0678.62070 · doi:10.2307/2289869 |
| [17] | Lee, A re-parametrization approach for dynamic space-time models, Journal of Statistical Theory and Practice 2 pp 1– (2008) · doi:10.1080/15598608.2008.10411856 |
| [18] | Lindstrom, Newton-Raphson and EM algorithms for linear mixed-effects models for repeated measures data, Journal of the American Statistical Association 83 pp 1014– (1988) · Zbl 0671.65119 · doi:10.2307/2290128 |
| [19] | Malm, Spatial and monthly trends in speciated fine particle concentration in the United States, Journal of Geophysical Research 109 (2004) · doi:10.1029/2003JD003739 |
| [20] | Miller, Subset Selection in Regression (2002) · Zbl 1051.62060 · doi:10.1201/9781420035933 |
| [21] | Morell, Linear transformations of linear mixed-effects models, The American Statistician 51 pp 338– (1997) · doi:10.2307/2685902 |
| [22] | Pauler, The Schwarz criterion and related methods for normal linear models, Biometrika 85 pp 13– (1998) · Zbl 1067.62550 · doi:10.1093/biomet/85.1.13 |
| [23] | Pinhiero, Unconstrained parameterizations for variance-covariance matrices, Statistics and Computing 6 pp 289– (1996) · doi:10.1007/BF00140873 |
| [24] | Pu, Selecting mixed-effects models based on generalized information criterion, Journal of Multivariate Analysis 97 pp 733– (2006) · Zbl 1085.62083 · doi:10.1016/j.jmva.2005.05.009 |
| [25] | Rao, A strongly consistent procedure for model selection in regression problems, Biometrika 76 pp 369– (1989) · Zbl 0669.62051 · doi:10.1093/biomet/76.2.369 |
| [26] | Schwarz, Estimating the dimension of a model, Annals of Statistics 6 pp 461– (1978) · Zbl 0379.62005 · doi:10.1214/aos/1176344136 |
| [27] | Shao, An asymptotic theory for linear model selection, Statistica Sinica 7 pp 221– (1997) · Zbl 1003.62527 |
| [28] | Smith, Parsimonious covariance matrix estimation for longitudinal data, Journal of the American Statistical Association 97 pp 1141– (2002) · Zbl 1041.62044 · doi:10.1198/016214502388618942 |
| [29] | Tibshirani, Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society, Series B 58 pp 267– (1996) · Zbl 0850.62538 |
| [30] | Vaida, Conditional Akaike information for mixed-effects models, Biometrika 92 pp 351– (2005) · Zbl 1094.62077 · doi:10.1093/biomet/92.2.351 |
| [31] | Wolfinger, Covariance structure selection in general mixed models, Communications in Statistics, Simulation and Computation 22 pp 1079– (1993) · Zbl 0800.62419 · doi:10.1080/03610919308813143 |
| [32] | Yang, Can the strengths of AIC and BIC be shared? A conflict between model identification and regression estimation, Biometrika 92 pp 937– (2005) · Zbl 1151.62301 · doi:10.1093/biomet/92.4.937 |
| [33] | Yu, An assessment of the ability of three-dimensional air quality models with current thermodynamic equilibrium models to predict aerosol NO-3, Journal of Geophysical Research 110 (2005) · doi:10.1029/2004JD004718 |
| [34] | Zou, The adaptive lasso and its oracle properties, Journal of the American Statistical Association 101 pp 1418– (2006) · Zbl 1171.62326 · doi:10.1198/016214506000000735 |
| [35] | Zou, Regularization and variable selection via the elastic net, Journal of the Royal Statistical Society, Series B 67 pp 301– (2005) · Zbl 1069.62054 · doi:10.1111/j.1467-9868.2005.00503.x |
| [36] | Zou, On the degrees of freedom of the lasso, Annals of Statistics 35 pp 2173– (2007) · Zbl 1126.62061 · doi:10.1214/009053607000000127 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
