Summary
Consider estimating the mean vector θ from dataN n (θ,σ 2 I) withl q norm loss,q≧1, when θ is known to lie in ann-dimensionall p ball,p∈(0, ∞). For largen, the ratio of minimaxlinear risk to minimax risk can bearbitrarily large ifp<q. Obvious exceptions aside, the limiting ratio equals 1 only ifp=q=2. Our arguments are mostly indirect, involving a reduction to a univariate Bayes minimax problem. Whenp<q, simple non-linear co-ordinatewise threshold rules are asymptotically minimax at small signal-to-noise ratios, and within a bounded factor of asymptotic minimaxity in general. We also give asymptotic evaluations of the minimax linear risk. Our results are basic to a theory of estimation in Besov spaces using wavelet bases (to appear elsewhere).
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Bickel, P.J.: Minimax estimation of the mean of a normal distribution when the parameter space is restricted. Ann. Stat.9, 1301–1309 (1981)
Birgé, L., Massart, P.: Rates of convergence for minimum contrast estimators. Technical Report Université Paris VI. Probab. Theory Relat. Fields97, 113–150 (1993)
Brown, L.D.: Admissible estimators, recurrent diffusions, and insoluble boundary value problems. Ann. Math. Stat.42, 855–903 (1971)
Brown, L.D., Johnstone, I.M., MacGibbon, K.B.: Variation Diminishing Transformations: A direct approach to total positivity and its statistical applications. J. Am. Stat. Assoc.76, 824–832 (1981)
Casella, G., Strawderman, W.E.: Estimating a bounded normal mean. Ann. Stat.9, 870–878 (1981)
Daubechies, I.: Orthonormal bases of compactly supported wavelets. Commun. Pure Appl. Math.41, 909–996 (1988)
Daubechies, I.: Ten Lectures on Wavelets SIAM: Philadelphia (1992)
Donoho, D.L.: Gelfandn-widths and the method of least squares. (Preprint, 1990)
Donoho, D.L.: Asymptotic minimax risk for sup-norm loss: solution via optimal recovery. Probab. Theory Relat. Fields (to appear, 1994)
Donoho, D.L., Johnstone, I.M.: Minimax risk overl p -balls. (Technical Report No. 322) Department of Statistics, Stanford: Stanford University 1989
Donoho, D.L., Johnstone, I.M.: Minimax risk overl p -balls forl q -error (Technical Report) Department of Statistics. Stanford: Stanford University 1992
Donoho, D.L., Johnstone, I.M.: Minimax estimation via wavelet shrinkage. (Technical Report). Ann. Stat. (to appear, 1995a)
Donoho, D.L., Johnstone, I.M.: Adapting to unknown smoothness via Wavelet shrinkage. J. Am. Stat. Assoc. (to appear, 1995b)
Donoho, D.L., Johnstone, I.M.: Non-classical Minimax Theorems, Thresholding, and Adaptation. Manuscript (1993)
Donoho, D.L., Johnstone, I.M.: Ideal Spatial Adaptation via Wavelet Shrinkage. Biometrika (to appear, 1994)
Donoho, D.L., Johnstone, I.M., Hoch, J.C., Stern, A.S.: Maximum Entropy and the Nearly Black Image. J. R. Stat. Soc. Ser. B54, 41���81 (1992) with discussion
Donoho, D.L., Johnstone, I.M., Kerkyacharian, G., Picard, D.: Wavelet Shrinkage: Asymptopia?
Donoho, D.J., Johnstone, I.M., Kerkyacharian, G., Picard, D.: Density estimation by wavelet thresholding. (Technical Report) Department of Statistics. Stanford: Stanford University 1993
Donoho, D.L., Liu, R.C., MacGibbon, K.B.: Minimax risk over hyperrectangles, and implications. Ann. Stat.18, 1416–1437 (1990)
Donoho, D.L., Liu, R.C.: Geometrizing Rates of Convergence, III. Ann. Stat.19, 668–701 (1991)
Feldman, I.: Constrained minimax estimation of the mean of the normal distribution with known variance. Ann. Stat.19, 2259–2265 (1991)
Huber, P.J.: Robust Estimation of a Location Parameter. Ann. Math. Stat.35, 73–101 (1964)
Huber, P.J.: Fisher Information and Spline Interpolation. Ann. Stat.2, 1029–1034 (1974)
Ibragimov, I.A., Khasminskii, R.Z.: Nonparametric estimation of the value of a linear functional in a Gaussian white noise. Theory Probab. Appl.29, 1–32 (1984)
Ibragimov, I.A., Hasminskii, R.Z.: On density estimation in the view of Kolmogorov's ideas in approximation theory. Ann. Stat.18, 999–1010 (1990)
Johnstone, I.M.: A moment inequality forL q estimation. Stat. Probab. Lett.12, 289–290 (1991)
Johnstone, I.M.: Minimax Bayes, asymptotic minimax and sparse wavelet priors. Statistical Decision Theory and Related Topics V (pp. 303–326). Berlin Heidelberg New York: Springer 1994
Karlin, S.: Total Positivity. Stanford: Stanford University Press 1968
Korostelev, A.P.: Asymptotic minimax estimation of regression function in the uniform norm. Teor. Veoryatn. Priment.37 (in russian). Theory Probab. Appl.37 (1993) (in english)
LeCam, L.: Asymptotic methods in statistical decision theory. Berlin Heidelberg New York: Springer 1986
Lehmann, E.L.: Theory of Point Estimation. New York: Wiley 1983
Lindenstrauss, J., Tzafiri, L.: The Classical Banach Spaces, II. Berlin Heidelberg New York: Springer 1979
Meyer, Y.: Ondelettes. Hermann: Paris 1990; Cambridge University Press 1993 (English translation)
Nemirovskii, A.S., Polyak, B.T., Tsybakov, A.B.: Rate of convergence of non-parametric estimates of maximum-likelihood type. Prob. Inf. Transm.21, 258–272 (1985)
Pinsker, M.S.: Optimal Filtration of square-integrable signals in Gaussian White Noise. Prob. Inf. Transm.16, 120–133 (1980)
Rockafellar, R.T.: Convex Analysis. Princeton: Princeton University Press 1970
Sacks, J., Strawderman, W.E.: Improving on linear minimax estimates. (Stat. Decis. Theory Relat. Topics III, vol. 2 pp. 287–304) J.R. Stat. Soc., Ser. B (with discussion) (to appear, 1995) Berlin Heidelberg New York: Springer 1982
Sion, M.: On general minimax theorems. Pac. J. Math.8, 171–176 (1958)
Van de Geer, S.: Estimating a regression function. Ann. Stat.18, 907–924 (1990)