Model selection and sharp asymptotic minimaxity. (English) Zbl 06176805
Summary: We obtain sharp minimax results for estimation of an \(n\)-dimensional normal mean under quadratic loss. The estimators are chosen by penalized least squares with a penalty that grows like \(ck\log(n/k)\), for \(k\) equal to the number of nonzero elements in the estimating vector. For a wide range of sparse parameter spaces, we show that the penalized estimator achieves the exact minimax rate with the correct multiplication constant if and only if \(c\) equals 2. Our results unify the theory obtained by many other authors for penalized estimation of normal means. In particular we establish that a conjecture by Abramovich et al. (Ann Stat 34:584-653, 2006) is true.
MSC:
| 62Gxx | Nonparametric inference |
Keywords:
FDR; minimax estimation; model selection; multiple comparisons; sharp asymptotic minimaxity; smoothing parameter selection; thresholding; wavelet denoising; waveletsReferences:
| [1] | Abramovich F., Benjamini Y., Donoho D.L., Johnstone I.M.: Adapting to unknown sparsity by controlling the false discovery rate. Ann. Stat. 34, 584-653 (2006) · Zbl 1092.62005 · doi:10.1214/009053606000000074 |
| [2] | Abramovich F., Grinshtein V., Pensky M.: On optimality of Bayesian testimation in the normal means problem. Ann. Stat. 35, 2261-2286 (2007) · Zbl 1126.62003 · doi:10.1214/009053607000000226 |
| [3] | Akaike, H.; Petrov, B. N. (ed.); Czáki, F. (ed.), Information theory and an extension of the maximum likelihood principle, 267-281 (1973), Budapest · Zbl 0283.62006 |
| [4] | Akaike H.: A new look at the statistical model identification. IEEE Trans. Automa. Control. 19(6), 716-723 (1974) · Zbl 0314.62039 · doi:10.1109/TAC.1974.1100705 |
| [5] | Barron A.R., Birgé L., Massart P.: Risk bounds for model selection via penalization. Probab. Theory. Relat. Fields 113, 301-413 (1999) · Zbl 0946.62036 · doi:10.1007/s004400050210 |
| [6] | Benjamini Y., Hochberg Y.: Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. R. Stat. Soc. B. 57, 289-300 (1995) · Zbl 0809.62014 |
| [7] | Benjamini Y., Gavrilov Y.: A simple forward selection procedure based on false discovery rate control. Ann. Appl. Stat. 3, 1, 179-198 (2009) · Zbl 1160.62068 · doi:10.1214/08-AOAS194 |
| [8] | Birgé l., Massart P.: Gaussian model selection. J. Eur. Math. Soc. 3, 203-268 (2001) · Zbl 1037.62001 · doi:10.1007/s100970100031 |
| [9] | Birgé L., Massart P.: Minimal penalties for Gaussian model selection. Probab. Theory. Relat. Fields 138, 33-73 (2007) · Zbl 1112.62082 · doi:10.1007/s00440-006-0011-8 |
| [10] | Brown L.D., Low M.G.: Asymptotic equivalence of nonparametric regression and white noise. Ann. Stat. 24, 2384-2398 (1996) · Zbl 0867.62022 · doi:10.1214/aos/1032181159 |
| [11] | Brown L.D., Cai T.T., Zhou H.H.: Robust nonparametric estimation via wavelet median regression. Ann. Stat. 36, 2055-2084 (2008) · Zbl 1148.62019 · doi:10.1214/07-AOS513 |
| [12] | Brown L.D., Cai T., Zhou H.H.: Nonparametric regression in exponential families. Ann. Stat. 38, 2005-2046 (2010) · Zbl 1202.62050 · doi:10.1214/09-AOS762 |
| [13] | Cai T.T., Zhou H.H.: Asymptotic equivalence and adaptive estimation for robust nonparametric regression. Ann. Stat. 37, 3204-3235 (2009) · Zbl 1191.62070 · doi:10.1214/08-AOS681 |
| [14] | Cörgő S., Mason D.: Central limit theorems for sums of extreme values. Math. Proc. Camb. 98, 547- (1985) · Zbl 0581.60025 · doi:10.1017/S0305004100063751 |
| [15] | Donoho D.L., Johnstone I.M., Hoch J.C., Stern A.S.: Maximum entropy and the nearly black object. J. R. Stat. Soc. B. 54(1), 41-81 (1992) · Zbl 0788.62103 |
| [16] | Donoho D.L., Johnstone I.M.: Ideal spatial adaptation by wavelet shrinkage. Biometrika 81(3), 425-455 (1994) · Zbl 0815.62019 · doi:10.1093/biomet/81.3.425 |
| [17] | Donoho D.L., Johnstone I.M.: Minimax Risk Over lp-Balls for lq-Error. Probab. Theory. Relat. Fields 99, 277-303 (1994) · Zbl 0802.62006 · doi:10.1007/BF01199026 |
| [18] | Donoho D.L., Johnstone I.M.: Adapting to unknown smoothness via wavelet Shrinkage. J. Am. Stat. Assoc. 90, 1200-1224 (1995) · Zbl 0869.62024 · doi:10.1080/01621459.1995.10476626 |
| [19] | Donoho D.L., Johnstone I.M.: Minimax estimation via wavelet shrinkage. Ann. Stat. 26, 879-921 (1998) · Zbl 0935.62041 · doi:10.1214/aos/1024691081 |
| [20] | David H.A., Nagaraja H.N.: Order Statistics, 3rd edn. Wiley & Sons, New York (2003) · Zbl 1053.62060 · doi:10.1002/0471722162 |
| [21] | Efron B.: Robbins, empirical Bayes and microarrays. Ann. Stat. 31, 366-378 (2003) · Zbl 1038.62099 · doi:10.1214/aos/1051027871 |
| [22] | Foster D.P., George E.I.: The risk inflation criterion for multiple regression. Ann. Stat. 22, 1947-1975 (1994) · Zbl 0829.62066 · doi:10.1214/aos/1176325766 |
| [23] | Foster D.P., Stine R.A.: Local asymptotic coding and the minimum description length. IEEE Trans. Inf. Theory 45, 1289-1293 (1999) · Zbl 0959.62006 · doi:10.1109/18.761287 |
| [24] | George E.I., Foster D.P.: Calibration and empirical Bayes variable selection. Biometrika 87, 731-747 (2000) · Zbl 1029.62008 · doi:10.1093/biomet/87.4.731 |
| [25] | Golubev G.K., Nussbaum M., Zhou H.H.: Asymptotic equivalence of spectral density estimation and Gaussian white noise. Ann. Stat. 38, 181-214 (2010) · Zbl 1181.62152 · doi:10.1214/09-AOS705 |
| [26] | Johnstone, I. M.; Gupta, S. (ed.); Berger, J. (ed.), Minimax Bayes asymptotic minimax and sparse wavelet priors, 303-326 (1994), Berlin · Zbl 0815.62017 · doi:10.1007/978-1-4612-2618-5_23 |
| [27] | Johnstone, I.M.: Gaussian Estimation: Sequence and Multiresolution Models. Unpublished manuscript. http://www-stat.stanford.edu/ imj/ (2011) · Zbl 0314.62039 |
| [28] | Nussbaum M.: Asymptotic equivalence of density estimation and Gaussian white noise. Ann. Stat. 24, 2399-2430 (1996) · Zbl 0867.62035 · doi:10.1214/aos/1032181160 |
| [29] | Tibshirani R., Knight K.: The covariance inflation criterion for adaptive model selection. J. R. Stat. Soc. B. 61, 529-546 (1999) · Zbl 0924.62031 · doi:10.1111/1467-9868.00191 |
| [30] | Yang Y., Barron A.R.: An asymptotic property of model selection criteria. IEEE Trans. Inf. Theory 44, 117-133 (1998) · Zbl 0938.62106 · doi:10.1109/18.650998 |
| [31] | Yang Y.: Model selection for nonparametric regression. Stat. Sinica. 9, 475-499 (1999) · Zbl 0921.62051 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
