Feature filter for estimating central mean subspace and its sparse solution. (English) Zbl 07422861
Summary: Sufficient dimension reduction, replacing the original predictors with a few linear combinations while keeping all the regression information, has been widely studied. A key goal is to find the central mean subspace, the intersection of all subspaces that provide such a reduction. To this end, a new sufficient dimension reduction method is proposed, with two estimation procedures, through a novel approach of feature filter, applicable to both univariate and multivariate responses. Asymptotic results are established. Estimation methods to determine the structural dimension, to obtain sparse estimator and to deal with large \(p\) small \(n\) data are provided. The efficacy of the method is demonstrated by simulations and a real data example.
MSC:
| 62G08 | Nonparametric regression and quantile regression |
| 62H12 | Estimation in multivariate analysis |
| 62J05 | Linear regression; mixed models |
| 62-08 | Computational methods for problems pertaining to statistics |
Keywords:
central mean subspace; characteristic function; feature filter; sufficient dimension reductionSoftware:
bootlibReferences:
| [1] | Aragon, Y., A Gauss implementation of multivariate sliced inverse regression, Comput. Stat., 12, 355-372 (1997) · Zbl 0885.62083 |
| [2] | Bura, E.; Cook, R. D., Extending sliced inverse regression: the weighted chi-squared test, J. Am. Stat. Assoc., 96, 996-1003 (2001) · Zbl 1047.62035 |
| [3] | Bura, E.; Cook, R. D., Estimating the structural dimension of regressions via parametric inverse regression, J. R. Stat. Soc. B, 63, 393-410 (2001) · Zbl 0979.62041 |
| [4] | Chen, X.; Zou, C.; Cook, R. D., Coordinate-independent sparse sufficient dimension reduction and variable selection, Ann. Stat., 38, 3696-3723 (2010) · Zbl 1204.62107 |
| [5] | Cook, R. D., On the interpretation of regression plots, J. Am. Stat. Assoc., 89, 177-190 (1994) · Zbl 0791.62066 |
| [6] | Cook, R. D., Graphics for regressions with a binary response, J. Am. Stat. Assoc., 91, 983-992 (1996) · Zbl 0882.62060 |
| [7] | Cook, R. D., Regression Graphics: Ideas for Studying Regressions Through Graphics (1998), Wiley: Wiley New York · Zbl 0903.62001 |
| [8] | Cook, R. D., Principal Hessian directions revisited (with discussion), J. Am. Stat. Assoc., 93, 84-100 (1998) · Zbl 0922.62057 |
| [9] | Cook, R. D., Fisher lecture: Dimension reduction in regression, Stat. Sci., 22, 1-26 (2007) · Zbl 1246.62148 |
| [10] | Cook, R. D.; Li, B., Dimension reduction for conditional mean in regression, Ann. Stat., 30, 455-474 (2002) · Zbl 1012.62035 |
| [11] | Cook, R. D.; Setodji, C. M., A model-free test for the reduced rank in multivariate regression, J. Am. Stat. Assoc., 98, 340-351 (2003) · Zbl 1041.62048 |
| [12] | Cook, R. D.; Weisberg, S., Discussion of “Sliced inverse regression for dimension reduction”, J. Am. Stat. Assoc., 86, 328-332 (1991) · Zbl 1353.62037 |
| [13] | Cook, R. D.; Yin, X., Dimension reduction and visualization in discriminant analysis (with discussion), Aust. N. Z. J. Stat., 43, 147-199 (2001) · Zbl 0992.62056 |
| [14] | Davison, A. C.; Hinkley, D. V., Bootstrap Methods and Their Application (1997), Cambridge · Zbl 0886.62001 |
| [15] | Fan, J.; Fan, Y., High dimensional classification using features annealed independence rules, Ann. Stat., 36, 2605-2637 (2008) · Zbl 1360.62327 |
| [16] | Fan, J.; Lv, J., Sure independence screening for ultrahigh dimensional feature space, J. R. Stat. Soc. B, 70, 849-911 (2008) · Zbl 1411.62187 |
| [17] | Fan, J.; Samworth, R.; Wu, Y., Ultrahigh dimensional feature selection: beyond the linear model, J. Mach. Learn. Res., 10, 2013-2038 (2009) · Zbl 1235.62089 |
| [18] | Fang, K.; Zhu, L. X., Asymptotics for kernel estimate of sliced inverse regression, Ann. Stat., 24, 1053-1068 (1996) · Zbl 0864.62027 |
| [19] | Hall, P.; Miller, H., Using generalized correlation to effect variable selection in very high dimensional problems, J. Comput. Graph. Stat., 18, 533-550 (2009) |
| [20] | Huang, J.; Horowitz, J.; Ma, S., Asymptotic properties of bridge estimators in sparse high-dimensional regression models, Ann. Stat., 36, 587-613 (2008) · Zbl 1133.62048 |
| [21] | Li, B.; Wang, S., On directional regression for dimension reduction, J. Am. Stat. Assoc., 102, 997-1008 (2007) · Zbl 1469.62300 |
| [22] | Li, B.; Wen, S.; Zhu, L. X., On a projective resampling method for dimension reduction with multivariate responses, J. Am. Stat. Assoc., 103, 1177-1186 (2008) · Zbl 1205.62067 |
| [23] | Li, B.; Zha, H.; Chiaromonte, F., Contour regression: a general approach to dimension reduction, Ann. Stat., 33, 1580-1616 (2005) · Zbl 1078.62033 |
| [24] | Li, K-C., Sliced inverse regression for dimension reduction, J. Am. Stat. Assoc., 86, 316-342 (1991) · Zbl 0742.62044 |
| [25] | Li, K-C., On principal Hessian direction for data visualization and dimension reduction: another application of Stein’s lemma, J. Am. Stat. Assoc., 87, 1025-1039 (1992) · Zbl 0765.62003 |
| [26] | Li, L., Sparse sufficient dimension reduction, Biometrika, 94, 603-613 (2007) · Zbl 1135.62062 |
| [27] | Li, R.; Zhong, W.; Zhu, L. P., Feature screening via distance correlation learning, J. Am. Stat. Assoc., 107, 1129-1139 (2012) · Zbl 1443.62184 |
| [28] | Luo, W.; Li, B., Combining eigenvalues and variation of eigenvectors for order determination, Biometrika, 103, 875-887 (2016) · Zbl 1506.62304 |
| [29] | Luo, W.; Li, B.; Yin, X., On efficient dimension reduction with respect to a statistical functional of interest, Ann. Stat., 42, 382-412 (2014) · Zbl 1285.62072 |
| [30] | Park, T.; Shao, X.; Yao, S., Partial martingale difference correlation, Electron. J. Stat., 9, 1492-1517 (2015) · Zbl 1329.62272 |
| [31] | Schott, J. R., Matrix Analysis for Statistics (2017), John Wiley: John Wiley New York · Zbl 1348.15001 |
| [32] | Setodji, C. M.; Cook, R. D., K-means inverse regression, Technometrics, 46, 421-429 (2004) |
| [33] | Shao, X.; Zhang, J., Martingale difference correlation and its use in high-dimension variable screening, J. Am. Stat. Assoc., 109, 1302-1318 (2014) · Zbl 1368.62157 |
| [34] | Sheng, W.; Yin, X., Direction estimation in single-index models via distance covariance, J. Multivar. Anal., 122, 148-161 (2013) · Zbl 1279.62097 |
| [35] | Székely, G. J.; Rizzo, M. L.; Bakirov, N. K., Measuring and testing independence by correlation of distances, Ann. Stat., 35, 2769-2794 (2007) · Zbl 1129.62059 |
| [36] | Wang, H.; Xia, Y., Sliced regression for dimension reduction, J. Am. Stat. Assoc., 103, 811-821 (2008) · Zbl 1306.62168 |
| [37] | Wang, Q.; Yin, X., A nonlinear multi-dimensional variable selection method for high dimensional data: sparse MAVE, Comput. Stat. Data Anal., 52, 4512-4520 (2008) · Zbl 1452.62136 |
| [38] | Wang, Q.; Yin, X.; Critchley, F., Dimension reduction based on the Hellinger integral, Biometrika, 102, 95-106 (2015) · Zbl 1345.62068 |
| [39] | Wu, W.; Yin, X., Stable estimation in dimension reduction, J. Comput. Graph. Stat., 24, 104-120 (2015) |
| [40] | Xia, Y.; Tong, H.; Li, W. K.; Zhu, L. X., An adaptive estimation of dimension reduction space, J. R. Stat. Soc. B, 64, 363-410 (2002) · Zbl 1091.62028 |
| [41] | Yang, B.; Yin, X.; Zhang, N., Sufficient variable selection using independence measures for continuous response, J. Multivar. Anal., 173, 480-493 (2019) · Zbl 1422.62208 |
| [42] | Ye, Z.; Weiss, R. E., Using the bootstrap to select one of a new class of dimension reduction methods, J. Am. Stat. Assoc., 98, 968-979 (2003) · Zbl 1045.62034 |
| [43] | Yin, X.; Bura, E., Moment-based dimension reduction for multivariate response regression, J. Stat. Plan. Inference, 136, 3675-3688 (2006) · Zbl 1093.62058 |
| [44] | Yin, X.; Cook, R. D., Dimension reduction for the conditional kth moment in regression, J. R. Stat. Soc. B, 64, 159-175 (2002) · Zbl 1067.62042 |
| [45] | Yin, X.; Cook, R. D., Direction estimation in single-index regressions, Biometrika, 92, 371-384 (2005) · Zbl 1094.62054 |
| [46] | Yin, X.; Hilafu, H., Sequential sufficient dimension reduction for large p, small n problems, J. R. Stat. Soc. B, 77, 879-892 (2015) · Zbl 1414.62194 |
| [47] | Yin, X.; Li, B., Sufficient dimension reduction based on an ensemble of minimum average variance estimators, Ann. Stat., 3392, 3416 (2011) · Zbl 1246.62141 |
| [48] | Yin, X.; Li, B.; Cook, R. D., Successive direction extraction for estimating the central subspace in a multiple-index regression, J. Multivar. Anal., 99, 1733-1757 (2008) · Zbl 1144.62030 |
| [49] | Yoo, J. K., A novel moment-based sufficient dimension reduction approach in multivariate regression, Comput. Stat. Data Anal., 52, 3843-3851 (2008) · Zbl 1452.62143 |
| [50] | Yoo, J. K., Sufficient dimension reduction through informative predictor subspace, Statistics, 55, 1086-1099 (2016) · Zbl 1356.62055 |
| [51] | Yoo, J. K.; Cook, R. D., Optimal sufficient dimension reduction for the conditional mean in multivariate regression, Biometrika, 94, 231-242 (2007) · Zbl 1142.62360 |
| [52] | Yoo, J. K.; Lee, K.; Wu, S., On the extension of sliced average variance estimation to multivariate regression, Stat. Methods Appl., 19, 529-540 (2010) · Zbl 1332.62219 |
| [53] | Zhu, L. X.; Ng, K., Asymptotics of sliced inverse regression, Stat. Sin., 5, 727-736 (1995) · Zbl 0824.62036 |
| [54] | Zhu, L. P.; Zhu, L. X., Dimension reduction for conditional variance in regressions, Stat. Sin., 19, 869-883 (2009) · Zbl 1166.62044 |
| [55] | Zhu, L. P.; Zhu, L. X.; Wen, S. Q., On dimension reduction in regressions with multivariate responses, Stat. Sin., 20, 1291-1307 (2010) · Zbl 1507.62267 |
| [56] | Zhu, Y.; Zeng, P., Fourier methods for estimating the central subspace and the central mean subspace in regression, J. Am. Stat. Assoc., 101, 1638-1651 (2006) · Zbl 1171.62325 |
| [57] | Zou, H., The adaptive lasso and its oracle properties, J. Am. Stat. Assoc., 101, 1418-1429 (2006) · Zbl 1171.62326 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
