Nonconcave penalized inverse regression in single-index models with high dimensional predictors. (English) Zbl 1157.62037
Summary: We aim to estimate the direction in general single-index models and to select important variables simultaneously when a diverging number of predictors are involved in regressions. Towards this end, we propose the nonconcave penalized inverse regression method. Specifically, the resulting estimation with the smoothly clipped absolute deviation (SCAD) penalty enjoys an oracle property in semi-parametric models even when the dimension, \(p_n\), of predictors goes to infinity. Under regularity conditions we also achieve the asymptotic normality when the dimension of the predictor vector goes to infinity at the rate of \(p_n=o(n^{1/3})\) where \(n\) is sample size, which enables us to construct confidence interval/region for the estimated index. The asymptotic results are augmented by simulations, and illustrated by analysis of an air pollution dataset.
MSC:
| 62H12 | Estimation in multivariate analysis |
| 62G20 | Asymptotic properties of nonparametric inference |
| 62G08 | Nonparametric regression and quantile regression |
| 62H15 | Hypothesis testing in multivariate analysis |
| 62P12 | Applications of statistics to environmental and related topics |
| 65C60 | Computational problems in statistics (MSC2010) |
References:
| [1] | Fan, J. Q.; Li, R., Variable selection via nonconcave penalized likelihood and its oracle properties, J. Amer. Statist. Assoc, 96, 1348-1360 (2001) · Zbl 1073.62547 |
| [2] | D.L. Donoho, High-dimensional data analysis: The curses and blessings of dimensionality, Aide-Memoire of a Lecture at AMS Conference on Math Challenges of the 21st Century, 2000; D.L. Donoho, High-dimensional data analysis: The curses and blessings of dimensionality, Aide-Memoire of a Lecture at AMS Conference on Math Challenges of the 21st Century, 2000 |
| [3] | Härdle, W.; Hall, P.; Ichimura, H., Optimal smoothing in single-index models, Ann. Statist., 21, 157-178 (1993) · Zbl 0770.62049 |
| [4] | Carroll, R. J.; Fan, J.; Gijbels, I.; Wand, M. P., Generalized partially linear single-index models, J. Amer. Statist. Assoc., 92, 477-489 (1997) · Zbl 0890.62053 |
| [5] | Härdle, W.; Stoker, T. M., Investigating smooth multiple regression by the method of average derivatives, J. Amer. Statist. Assoc., 84, 986-995 (1989) · Zbl 0703.62052 |
| [6] | Xia, Y. C.; Tong, H.; Li, W. K.; Zhu, L. X., An adaptive estimation of optimal regression subspace, J. Roy. Statist. Soc. B, 64, 363-410 (2002) · Zbl 1091.62028 |
| [7] | Altham, P. M.E, Improving the precision of estimation by fitting a generalized linear model and quasi-likelihood, J. Roy. Statist. Soc. B., 46, 118-119 (1984) |
| [8] | Kong, E.; Xia, Y. C., Variable selction for the single-index model, Biometrika, 94, 217-229 (2007) · Zbl 1142.62353 |
| [9] | Li, K. C.; Duan, N. H., Regression analysis under link violation, Ann. Statist., 17, 1009-1052 (1989) · Zbl 0753.62041 |
| [10] | Li, K. C., Sliced inverse regression for dimension reduction (with discussion), J. Amer. Statist. Assoc., 86, 316-342 (1991) · Zbl 0742.62044 |
| [11] | Cook, R. D., Regression Graphics: Ideas for Studying Regressions Through Graphics (1998), Wiley & Sons: Wiley & Sons New York · Zbl 0903.62001 |
| [12] | Cook, R. D.; Ni, L., Sufficient dimension reduction via inverse regression: A minimum discrepancy approach, J. Amer. Statist. Assoc., 100, 410-428 (2005) · Zbl 1117.62312 |
| [13] | Cook, R. D.; Weisberg, S., Discussion to “Sliced inverse regression for dimension reduction”, J. Amer. Statist. Assoc., 86, 316-342 (1991) · Zbl 0742.62044 |
| [14] | Li, B.; Zha, H.; Chiaromonte, F., Contour regression: A general approach to dimension reduction, Ann. Statist., 33, 1580-1616 (2005) · Zbl 1078.62033 |
| [15] | Li, B.; Wang, S. L., On directional regression for dimension reduction, J. Amer. Statist. Assoc., 102, 997-1008 (2007) · Zbl 1469.62300 |
| [16] | Chen, C. H.; Li, K. C., Can SIR be as popular as multiple linear regression?, Statist. Sinica, 8, 289-316 (1998) · Zbl 0897.62069 |
| [17] | Cook, R. D., Testing predictor contributions in sufficient dimension reduction, Ann. Statist., 32, 1061-1092 (2004) · Zbl 1092.62046 |
| [18] | Ni, L.; Cook, R. D.; Tsai, C. L., A note on shrinkage sliced inverse regression, Biometrika, 92, 242-247 (2005) · Zbl 1068.62080 |
| [19] | Li, L. X.; Nachtsheim, C. J., Sparse sliced inverse regression, Technometrics, 48, 503-510 (2006) |
| [20] | Li, L. X., Sparse sufficient dimension reduction, Biometrika, 92, 603-613 (2007) · Zbl 1135.62062 |
| [21] | Donoho, D. L.; Johnstone, I. M., Ideal spatial adaptation by wavelet shrinkage, Biometrika, 81, 425-455 (1994) · Zbl 0815.62019 |
| [22] | Fan, J. Q.; Peng, H., Nonconcave penalized likelihood with a diverging number of parameters, Ann. Statist., 32, 928-961 (2004) · Zbl 1092.62031 |
| [23] | Fan, J. Q., Commments on “Wavelets in Statistics: A review” by A. Antoniadis, J. Amer. Statist. Assoc., 6, 131-138 (1997) · Zbl 1454.62116 |
| [24] | Hall, P.; Li, K. C., On almost linearity of low dimensional projection from high dimensional data, Ann. Statist., 21, 867-889 (1993) · Zbl 0782.62065 |
| [25] | Diaconis, P.; Freedman, D., Asymptotics of graphical projection pursuit, Ann. Statist., 12, 793-815 (1984) · Zbl 0559.62002 |
| [26] | Huber, P. J., Robust regression: Asymptotics, conjectures and Monte Carlo, Ann. Statist., 1, 799-821 (1973) · Zbl 0289.62033 |
| [27] | Bickel, P.; Klaassen, C. A.J.; Ritov, Y.; Wellner, J., Efficient and Adaptive Inference in Semi-parametric Models (1993), John Hopkins University Press: John Hopkins University Press Baltimore · Zbl 0786.62001 |
| [28] | Leeb; Poetscher, Sparse estimators and the oracle property, or the return to the Hodges’ estimator, J. Econometrics, 142, 201-211 (2008) · Zbl 1418.62272 |
| [29] | Pollard, D., Convergence of Stochastic Processes (1980), Springer-Verlag: Springer-Verlag New York · Zbl 0425.62029 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
