Variable selection using penalized empirical likelihood. (English) Zbl 1234.62030
Summary: This paper considers variable selection for moment restriction models. We propose a penalized empirical likelihood (PEL) approach that has desirable asymptotic properties comparable to the penalized likelihood approach, which relies on a correct parametric likelihood specification. In addition to being consistent and having the oracle property, PEL admits inference on the parameters without having to estimate the estimators’ covariance. An approximate algorithm, along with a consistent BIC-type criterion for selecting the tuning parameters, is provided for PEL. The proposed algorithm enjoys considerable computational efficiency and overcomes the drawbacks of the local quadratic approximation of nonconcave penalties. Simulation studies to evaluate and compare the performance of our method with those of the existing ones show that PEL is competitive and robust. The proposed method is illustrated with two real examples.
MSC:
| 62G05 | Nonparametric estimation |
| 62F07 | Statistical ranking and selection procedures |
| 65C60 | Computational problems in statistics (MSC2010) |
Software:
ElemStatLearnReferences:
| [1] | Akaike H. Information theory and an extension of the maximum likelihood principle. In: Petrov B N, Csaki F, eds. Proceedings of the 2nd International Symposium Information Theory. Budapest: Akademia Kiado, 1973, 267–281 · Zbl 0283.62006 |
| [2] | Breiman L. Heuristics of instability and stabilization in model selection. Ann Statist, 1996, 24: 2350–2383 · Zbl 0867.62055 · doi:10.1214/aos/1032181158 |
| [3] | Bühlmann P, Meier L. Discussion: one-step sparse estimates in nonvoncave penalized likelihood models. Ann Statist, 2008, 36: 1534–1541 · Zbl 1282.62096 · doi:10.1214/07-AOS0316A |
| [4] | Cai J, Fan J, Li R, et al. Variable selection for multivariate failure time data. Biometrika, 2005, 92: 303–316 · Zbl 1094.62123 · doi:10.1093/biomet/92.2.303 |
| [5] | Candes E, Tao T. The dantzig selector: Statistical estimation when p is much larger than n. Ann Statist, 2007, 35: 2313–2351 · Zbl 1139.62019 · doi:10.1214/009053606000001523 |
| [6] | Chen J, Variyath A M, Abraham B. Adjusted empirical likelihood and its properties. J Comput Graph Statist, 2008, 17: 426–443 · doi:10.1198/106186008X321068 |
| [7] | Efron B, Hastie T, Johnstone I, et al. Least angle regression (with discussion, and a rejoinder by the authors). Ann Statist, 2004, 32: 407–499 · Zbl 1091.62054 · doi:10.1214/009053604000000067 |
| [8] | Fan J, Li R. Variable selection via nonconcave penalized likelihood and its oracle properties. J Amer Statist Assoc, 2001, 96: 1348–1360 · Zbl 1073.62547 · doi:10.1198/016214501753382273 |
| [9] | Fan J, Li R. Variable selection for Cox’s proportional hazards model and frailty model. Ann Statist, 2002, 30: 74–99 · Zbl 1012.62106 · doi:10.1214/aos/1015362185 |
| [10] | Fan J, Li R. New estimation and model selection procedures for semiparametric modeling in longitudinal data analysis. J Amer Statist Assoc, 2004, 99: 710–723 · Zbl 1117.62329 · doi:10.1198/016214504000001060 |
| [11] | Fan J, Li R. Statistical challenges with high dimensionality: Feature selection in knowledge discovery. In: Proceedings of the Madrid International Congress of Mathematicians, 2006 · Zbl 1117.62137 |
| [12] | Fan J, Peng H. On non-concave penalized likelihood with diverging number of parameters. Ann Statist, 2004, 32: 928–961 · Zbl 1092.62031 · doi:10.1214/009053604000000256 |
| [13] | Fan J, Gijbels I. Local Polynomial Modelling and Its Applications. New York: Chapman and Hall, 1996 · Zbl 0873.62037 |
| [14] | Foster D, George E. The risk inflation criterion for multiple regression. Ann Statist, 1994, 22: 1947–1975 · Zbl 0829.62066 · doi:10.1214/aos/1176325766 |
| [15] | Frank I, Friedman J. A statistical view of some chemometrics regression tools. Technometrics, 1993, 35: 109–148 · Zbl 0775.62288 · doi:10.1080/00401706.1993.10485033 |
| [16] | Freund Y, Schapire R E. A decision-theoretic generalization of online learning and an application to boosting. J Comput System Sci, 1997, 55: 119–139 · Zbl 0880.68103 · doi:10.1006/jcss.1997.1504 |
| [17] | Fu W. Penalized regression: The bridge versus the Lasso. J Comput Graph Statist, 1998, 7: 397–416 |
| [18] | Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning: Data Mining, Inference and Prediction. New York: Springer Verlag, 2001 · Zbl 0973.62007 |
| [19] | Hesterberg T, Choi N, Meier L, et al. Least angle and L 1 penalized regression: A review. Statistics Survey, 2008, 2: 61–93 · Zbl 1189.62070 · doi:10.1214/08-SS035 |
| [20] | Hoerl A, Kennard R. Ridge regression. In: Encyclopedia of Statistical Sciences, vol. 8. New York: Wiley, 1988, 129–136 |
| [21] | Hunter D, Li R. Variable selection using MM algorithms. Ann Statist, 2005, 33: 1617–1642 · Zbl 1078.62028 · doi:10.1214/009053605000000200 |
| [22] | Khalili A, Chen J. Variable selection in finite mixture of regression models. J Amer Statist Assoc, 2007, 102: 1025–1038 · Zbl 1469.62306 · doi:10.1198/016214507000000590 |
| [23] | Li R, Liang H. Variable selection in semiparametric regression modeling. Ann Statist, 2008, 36: 261–286 · Zbl 1132.62027 · doi:10.1214/009053607000000604 |
| [24] | Lv J, Fan Y. A unified approach to model selection and sparse recovery using regularized least squares. Ann Statist, 2009, 37: 3498–3528 · Zbl 1369.62156 · doi:10.1214/09-AOS683 |
| [25] | Nierenberg D W, Stukel T A, Baron J A, et al. Determinants of plasma levels of beta-carotene and retinol. Amer J Epidemiol, 1989, 130: 511–521 |
| [26] | Owen A. Empirical likelihood ratio confidence intervals for a single functional. Biometrika, 1988, 75: 237–249 · Zbl 0641.62032 · doi:10.1093/biomet/75.2.237 |
| [27] | Owen A. Empirical likelihood ratio confidence regions. Ann Statist, 1990, 18: 90–120 · Zbl 0712.62040 · doi:10.1214/aos/1176347494 |
| [28] | Owen A. Empirical likelihood for linear models. Ann Statist, 1991, 19: 1725–1747 · Zbl 0799.62048 · doi:10.1214/aos/1176348368 |
| [29] | Owen A. Empirical Likelihood. Boca Raton, FL: Chapman & Hall/CRC Press, 2001 |
| [30] | Park M Y, Hastie T. L 1 regularization-path algorithm for generalized linear models. J R Stat Soc Ser B Stat Methodol, 2007, 69: 659–677 · doi:10.1111/j.1467-9868.2007.00607.x |
| [31] | Qin J, Lawless J. Empirical likelihood and general estimating equations. Ann Statist, 1994, 22: 300–325 · Zbl 0799.62049 · doi:10.1214/aos/1176325370 |
| [32] | Schwarz G. Estimating the dimension of a model. Ann Statist, 1978, 6: 461–464 · Zbl 0379.62005 · doi:10.1214/aos/1176344136 |
| [33] | Tibshirani R J. Regression shrinkage and selection via the Lasso. J R Stat Soc Ser B Stat Methodol, 1996, 58: 267–288 · Zbl 0850.62538 |
| [34] | Variyath A M, Chen J, Abraham B. Empirical likelihood based variable selection. J Statist Plann Inference, 2010, 80: 971–981 · Zbl 1179.62047 · doi:10.1016/j.jspi.2009.09.025 |
| [35] | Wang H, Leng C. Unified Lasso estimation by least squares approximation. J Amer Statist Assoc, 2007, 102: 1039–1048 · Zbl 1306.62167 · doi:10.1198/016214507000000509 |
| [36] | Wang H, Li R, Tsai C H. Tuning parameter selectors for the smoothly clipped absolute deviation method. Biometrika, 2007, 94: 553–568 · Zbl 1135.62058 · doi:10.1093/biomet/asm053 |
| [37] | Wang L, Li R. Weighted wilcoxon-type smoothly clipped absolute deviation method. Biometrics, 2009, 65: 564–571 · Zbl 1167.62093 · doi:10.1111/j.1541-0420.2008.01099.x |
| [38] | Yuan M, Lin Y. Model selection and estimation in regression with grouped variables. J R Stat Soc Ser B Stat Methodol, 2006, 68: 49–68 · Zbl 1141.62030 · doi:10.1111/j.1467-9868.2005.00532.x |
| [39] | Zhang C H. Penalized linear unbiased selection. Ann Statist, 2010, 38: 894–942 · Zbl 1183.62120 · doi:10.1214/09-AOS729 |
| [40] | Zou H. The adaptive Lasso and its oracle properties. J Amer Statist Assoc, 2006, 101: 1418–1429 · Zbl 1171.62326 · doi:10.1198/016214506000000735 |
| [41] | Zou H. Hastie T. Regularization and variable selection via the elastic net. J R Stat Soc Ser B Stat Methodol, 2005, 67: 301–320 · Zbl 1069.62054 · doi:10.1111/j.1467-9868.2005.00503.x |
| [42] | Zou H, Li R. One-step sparse estimates in nonconcave penalized likelihood models. Ann Statist, 2008, 36: 1509–1533 · Zbl 1142.62027 · doi:10.1214/009053607000000802 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
