Penalized least squares for single index models. (English) Zbl 1204.62070
Summary: The single index model is a useful regression model. We propose a nonconcave penalized least squares method to estimate both the parameters and the link function of a single index model. Compared to other variable selection and estimation methods, the proposed method can estimate parameters and select variables simultaneously. When the dimension of parameters in the single index model is a fixed constant, under some regularity conditions, we demonstrate that the proposed estimators for the parameters have the so-called oracle property, and furthermore we establish the asymptotic normality and develop a sandwich formula to estimate the standard deviations of the proposed estimators. Simulation studies and a real data analysis are presented to illustrate the proposed methods.
MSC:
| 62G08 | Nonparametric regression and quantile regression |
| 62H12 | Estimation in multivariate analysis |
| 62G20 | Asymptotic properties of nonparametric inference |
| 65C60 | Computational problems in statistics (MSC2010) |
Keywords:
local polynomial regression; nonconcave penalized least squares; SCAD penalty; single index model; variable selectionReferences:
| [1] | Bai, Z. D.; Rao, C. R.; Wu, Y., Model selection with data-oriented penalty, Journal of Statistical Planning and Inference, 77, 103-117 (1999) · Zbl 0926.62045 |
| [2] | Carroll, R. J.; Fan, J.; Gijbels, I.; Wand, M. P., Generalized partially linear single-index models, Journal of the American Statistical Association, 92, 477-489 (1997) · Zbl 0890.62053 |
| [3] | Cook, D. R.; Weisberg, S., Sliced inverse regression for dimension reduction: comment, Journal of the American Statistical Association, 86, 328-332 (1991) · Zbl 1353.62037 |
| [4] | Duan, N.; Li, K.-C., Slicing regression: a link-free regression method, Annals of Statistics, 19, 505-530 (1991) · Zbl 0738.62070 |
| [5] | Fan, J., Comments on ‘Wavelets in Statistics: A Review’ by A. Antoniadis, Journal of the Italian Statistical Association, 6, 131-138 (1997) · Zbl 1454.62116 |
| [6] | Fan, J.; Gijbels, I., Local Polynomial Modelling and its Applications (1996), Chapman & Hall: Chapman & Hall London · Zbl 0873.62037 |
| [7] | Fan, J.; Li, R., Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association, 96, 1348-1360 (2001) · Zbl 1073.62547 |
| [8] | Fan, J.; Li, R., Variable selection for Cox’s proportional hazards model and frailty model, Annals of Statistics, 30, 74-99 (2002) · Zbl 1012.62106 |
| [9] | Fan, J.; Peng, H., Nonconcave penalized likelihood with a diverging number of parameters, Annals of Statistics, 32, 928-961 (2004) · Zbl 1092.62031 |
| [10] | Fan, J.; Yao, Q., Nonlinear Time Series (2003), Springer · Zbl 1014.62103 |
| [11] | Härdle, W.; Hall, P.; Ichimura, H., Optimal smoothing in single-index models, Annals of Statistics, 21, 157-178 (1993) · Zbl 0770.62049 |
| [12] | Härdle, W.; Tsybakov, A. B., How sensitive are average derivatives, Journal Econometrics, 58, 31-48 (1993) · Zbl 0772.62021 |
| [13] | Hristache, M.; Juditsky, A.; Spokoiny, V., Direct estimation of the index coefficient in a single-index model, Annals of Statistics, 29, 595-623 (2001) · Zbl 1012.62043 |
| [14] | Horowitz, J. L.; Härdle, W., Direct semiparametric estimation of single-index models with discrete covariates, Journal of the American Statistical Association, 91, 1632-1640 (1996) · Zbl 0881.62037 |
| [15] | Hunter, D.; Li, R., Variable selection using MM algorithms, Annals of Statistics, 33, 1617-1642 (2005) · Zbl 1078.62028 |
| [16] | Ichimura, H., Semiparametric least square (SLS) and weighted SLS estimation of single-index models, Journal of Econometrics, 58, 71-120 (1993) · Zbl 0816.62079 |
| [17] | Katch, F.; McArdle, W., Nutrition, Weight Control, and Exercise (1977), Houghton Mifflin Co.: Houghton Mifflin Co. Boston |
| [18] | Kong, E.; Xia, Y., Variable selection for the single-index model, Biometrika, 94, 217-229 (2007) · Zbl 1142.62353 |
| [19] | Li, R.; Liang, H., Variable selection in semiparametric regression modeling, Annals of Statistics, 36, 261-286 (2008) · Zbl 1132.62027 |
| [20] | Naik, P. A.; Tsai, C.-L., Single-index model selections, Biometrika, 88, 821-832 (2001) · Zbl 0988.62042 |
| [21] | Penrose, K. W.; Nelson, A. G.; Fisher, A. G., Generalized body composition prediction equation for men using simple measurement techniques, Medicine and Science in Sports and Exercise, 17, 189 (1985) |
| [22] | Powell, J. L.; Stock, J. M.; Stoker, T. M., Semiparametric estimation of index coefficients, Econometrica, 57, 1403-1430 (1989) · Zbl 0683.62070 |
| [23] | Quarteroni, A.; Sacco, R.; Saleri, F., Numerical Mathematics (2000), Springer-Verlag: Springer-Verlag New York · Zbl 0943.65001 |
| [24] | Ruppert, D.; Sheather, S. J.; Wand, M. P., An effective bandwidth selector for local least square regression, Journal of the American Statistical Association, 90, 1257-1270 (1995) · Zbl 0868.62034 |
| [25] | Siri, W. E., Gross composition of the body, Advances in Biological and Medical Physics, IV (1956), Academic Press, Inc.: Academic Press, Inc. New York |
| [26] | Stoker, T. M., Consistent estimation of scale coefficients, Econometrica, 54, 1461-1481 (1986) · Zbl 0628.62105 |
| [27] | Tibshirani, R. J., Regression shrinkage and selection via the LASSO, Journal of the Royal Statistical Society, Series B, 58, 267-288 (1996) · Zbl 0850.62538 |
| [28] | van de Geer, S. A., Empirical Processes in M-estimation (2000), Cambridge University Press · Zbl 1179.62073 |
| [29] | Wang, H.; Li, R.; Tsai, C.-L., Tuning parameter selectors for the smoothly clipped absolute deviation method, Biometrika, 94, 553-568 (2007) · Zbl 1135.62058 |
| [30] | Zhu, L. P.; Zhu, L. X., Nonconcave penalized inverse regression in single-index models with high dimensional predictors, Journal of Multivariate Analysis, 100, 862-875 (2009) · Zbl 1157.62037 |
| [31] | Zou, H., The adaptive lasso and its oracle properties, Journal of the American Statistical Association, 101, 1418-1429 (2006) · Zbl 1171.62326 |
| [32] | Zou, H.; Li, R., One-step sparse estimates in nonconcave penalized likelihood models, Annals of Statistics, 36, 1509-1533 (2008) · Zbl 1142.62027 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
