×

Adaptively weighted group Lasso for semiparametric quantile regression models. (English) Zbl 1429.62140

Summary: We propose an adaptively weighted group Lasso procedure for simultaneous variable selection and structure identification for varying coefficient quantile regression models and additive quantile regression models with ultra-high dimensional covariates. Under a strong sparsity condition, we establish selection consistency of the proposed Lasso procedure when the weights therein satisfy a set of general conditions. This consistency result, however, is reliant on a suitable choice of the tuning parameter for the Lasso penalty, which can be hard to make in practice. To alleviate this difficulty, we suggest a BIC-type criterion, which we call high-dimensional information criterion (HDIC), and show that the proposed Lasso procedure with the tuning parameter determined by HDIC still achieves selection consistency. Our simulation studies support strongly our theoretical findings.

MSC:

62G08 Nonparametric regression and quantile regression
62J07 Ridge regression; shrinkage estimators (Lasso)
62G20 Asymptotic properties of nonparametric inference

References:

[1] Belloni, A. and Chernozhukov, V. (2011). \( \ell_1\) -penalized quantile regression in high-dimensional sparse models. Ann. Statist.39 82-130. · Zbl 1209.62064 · doi:10.1214/10-AOS827
[2] Bickel, P.J., Ritov, Y. and Tsybakov, A.B. (2009). Simultaneous analysis of lasso and Dantzig selector. Ann. Statist.37 1705-1732. · Zbl 1173.62022 · doi:10.1214/08-AOS620
[3] Bühlmann, P. and van de Geer, S. (2011). Statistics for High-Dimensional Data: Methods, Theory and Applications. Springer Series in Statistics. Heidelberg: Springer. · Zbl 1273.62015
[4] Cai, Z. and Xiao, Z. (2012). Semiparametric quantile regression estimation in dynamic models with partially varying coefficients. J. Econometrics167 413-425. · Zbl 1441.62623 · doi:10.1016/j.jeconom.2011.09.025
[5] Chen, J. and Chen, Z. (2008). Extended Bayesian information criteria for model selection with large model spaces. Biometrika95 759-771. · Zbl 1437.62415 · doi:10.1093/biomet/asn034
[6] Cheng, M.-Y., Honda, T., Li, J. and Peng, H. (2014). Nonparametric independence screening and structure identification for ultra-high dimensional longitudinal data. Ann. Statist.42 1819-1849. · Zbl 1305.62169 · doi:10.1214/14-AOS1236
[7] Fan, J., Fan, Y. and Barut, E. (2014). Adaptive robust variable selection. Ann. Statist.42 324-351. · Zbl 1296.62144 · doi:10.1214/13-AOS1191
[8] Fan, J., Feng, Y. and Song, R. (2011). Nonparametric independence screening in sparse ultra-high-dimensional additive models. J. Amer. Statist. Assoc.106 544-557. · Zbl 1232.62064 · doi:10.1198/jasa.2011.tm09779
[9] Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc.96 1348-1360. · Zbl 1073.62547 · doi:10.1198/016214501753382273
[10] Fan, J., Ma, Y. and Dai, W. (2014). Nonparametric independence screening in sparse ultra-high-dimensional varying coefficient models. J. Amer. Statist. Assoc.109 1270-1284. · Zbl 1368.62095 · doi:10.1080/01621459.2013.879828
[11] Fan, J. and Song, R. (2010). Sure independence screening in generalized linear models with NP-dimensionality. Ann. Statist.38 3567-3604. · Zbl 1206.68157 · doi:10.1214/10-AOS798
[12] Fan, J., Xue, L. and Zou, H. (2014). Strong oracle optimality of folded concave penalized estimation. Ann. Statist.42 819-849. · Zbl 1305.62252 · doi:10.1214/13-AOS1198
[13] Fan, J., Xue, L. and Zou, H. (2016). Multitask quantile regression under the transnormal model. J. Amer. Statist. Assoc.111 1726-1735.
[14] Hastie, T., Tibshirani, R. and Wainwright, M. (2015). Statistical Learning with Sparsity: The Lasso and Generalizations. Monographs on Statistics and Applied Probability143. Boca Raton, FL: CRC Press. · Zbl 1319.68003
[15] He, X., Wang, L. and Hong, H.G. (2013). Quantile-adaptive model-free variable screening for high-dimensional heterogeneous data. Ann. Statist.41 342-369. · Zbl 1295.62053 · doi:10.1214/13-AOS1087
[16] Honda, T., Ing, C.-K. and Wu, W.-Y. (2019). Supplement to “Adaptively weighted group Lasso for semiparametric quantile regression models.” DOI:10.3150/18-BEJ1091SUPP. · Zbl 1429.62140
[17] Honda, T. and Yabe, R. (2017). Variable selection and structure identification for varying coefficient Cox models. J. Multivariate Anal.161 103-122. · Zbl 1373.62157 · doi:10.1016/j.jmva.2017.07.007
[18] Ing, C.-K. and Lai, T.L. (2011). A stepwise regression method and consistent model selection for high-dimensional sparse linear models. Statist. Sinica21 1473-1513. · Zbl 1225.62095 · doi:10.5705/ss.2010.081
[19] Kato, K. (2011). Group lasso for high dimensional sparse quantile regression models. ArXiv preprint. Available at arXiv:1103.1458.
[20] Kim, Y. and Jeon, J.-J. (2016). Consistent model selection criteria for quadratically supported risks. Ann. Statist.44 2467-2496. · Zbl 1365.60030 · doi:10.1214/15-AOS1413
[21] Lee, E.R. and Mammen, E. (2016). Local linear smoothing for sparse high dimensional varying coefficient models. Electron. J. Stat.10 855-894. · Zbl 1349.62313 · doi:10.1214/16-EJS1110
[22] Lee, E.R., Noh, H. and Park, B.U. (2014). Model selection via Bayesian information criterion for quantile regression models. J. Amer. Statist. Assoc.109 216-229. · Zbl 1367.62122 · doi:10.1080/01621459.2013.836975
[23] Lian, H. (2012). Semiparametric estimation of additive quantile regression models by two-fold penalty. J. Bus. Econom. Statist.30 337-350.
[24] Lian, H., Liang, H. and Ruppert, D. (2015). Separation of covariates into nonparametric and parametric parts in high-dimensional partially linear additive models. Statist. Sinica25 591-607. · Zbl 1534.62053
[25] Lv, S., Lin, H., Lian, H. and Huang, J. (2018). Oracle inequalities for sparse additive quantile regression in reproducing kernel Hilbert space. Ann. Statist.46 781-813. · Zbl 1459.62053 · doi:10.1214/17-AOS1567
[26] Ma, S. and He, X. (2016). Inference for single-index quantile regression models with profile optimization. Ann. Statist.44 1234-1268. · Zbl 1338.62119 · doi:10.1214/15-AOS1404
[27] Meier, L., van de Geer, S. and Bühlmann, P. (2008). The group Lasso for logistic regression. J. R. Stat. Soc. Ser. B. Stat. Methodol.70 53-71. · Zbl 1400.62276 · doi:10.1111/j.1467-9868.2007.00627.x
[28] Sherwood, B. and Wang, L. (2016). Partially linear additive quantile regression in ultra-high dimension. Ann. Statist.44 288-317. · Zbl 1331.62264 · doi:10.1214/15-AOS1367
[29] Tang, Y., Song, X., Wang, H.J. and Zhu, Z. (2013). Variable selection in high-dimensional quantile varying coefficient models. J. Multivariate Anal.122 115-132. · Zbl 1279.62049 · doi:10.1016/j.jmva.2013.07.015
[30] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B58 267-288. · Zbl 0850.62538 · doi:10.1111/j.2517-6161.1996.tb02080.x
[31] van de Geer, S. (2016). Estimation and Testing Under Sparsity. Lecture Notes in Math.2159. Cham: Springer. Lecture notes from the 45th Probability Summer School held in Saint-Four, 2015, École d’Été de Probabilités de Saint-Flour. [Saint-Flour Probability Summer School].
[32] Wang, H. (2009). Forward regression for ultra-high dimensional variable screening. J. Amer. Statist. Assoc.104 1512-1524. · Zbl 1205.62103 · doi:10.1198/jasa.2008.tm08516
[33] Wang, H.J., Zhu, Z. and Zhou, J. (2009). Quantile regression in partially linear varying coefficient models. Ann. Statist.37 3841-3866. · Zbl 1191.62077 · doi:10.1214/09-AOS695
[34] Yan, J. and Huang, J. (2012). Model selection for Cox models with time-varying coefficients. Biometrics68 419-428. · Zbl 1251.62052 · doi:10.1111/j.1541-0420.2011.01692.x
[35] Yuan, M. and Lin, Y. (2006). Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B. Stat. Methodol.68 49-67. · Zbl 1141.62030 · doi:10.1111/j.1467-9868.2005.00532.x
[36] Zhang, H.H., Cheng, G. and Liu, Y. (2011). Linear or nonlinear? Automatic structure discovery for partially linear models. J. Amer. Statist. Assoc.106 1099-1112. · Zbl 1229.62051 · doi:10.1198/jasa.2011.tm10281
[37] Zheng, Q., Peng, L. and He, X. (2015). Globally adaptive quantile regression with ultra-high dimensional data. Ann. Statist.43 2225-2258. · Zbl 1327.62424 · doi:10.1214/15-AOS1340
[38] Zhu, L., Huang, M. and Li, R. (2012). Semiparametric quantile regression with high-dimensional covariates. Statist. Sinica22 1379-1401. · Zbl 1253.62032
[39] Zou, H. (2006). The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc.101 1418-1429. · Zbl 1171.62326 · doi:10.1198/016214506000000735
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.