×

Global risk bounds and adaptation in univariate convex regression. (English) Zbl 1327.62255

Summary: We consider the problem of nonparametric estimation of a convex regression function \(\phi _0\). We study the risk of the least squares estimator (LSE) under the natural squared error loss. We show that the risk is always bounded from above by \(n^{-4/5}\) modulo logarithmic factors while being much smaller when \(\phi_0\) is well-approximable by a piecewise affine convex function with not too many affine pieces (in which case, the risk is at most \(1/n\) up to logarithmic factors). On the other hand, when \(\phi_0\) has curvature, we show that no estimator can have risk smaller than a constant multiple of \(n^{-4/5}\) in a very strong sense by proving a “local” minimax lower bound. We also study the case of model misspecification where we show that the LSE exhibits the same global behavior provided the loss is measured from the closest convex projection of the true regression function. In the process of deriving our risk bounds, we prove new results for the metric entropy of local neighborhoods of the space of univariate convex functions. These results, which may be of independent interest, demonstrate the non-uniform nature of the space of univariate convex functions in sharp contrast to classical function spaces based on smoothness constraints.

MSC:

62G08 Nonparametric regression and quantile regression
62C20 Minimax procedures in statistical decision theory

Software:

LogConcDEAD

References:

[1] Atkinson, K.E.: An Introduction to Numerical Analysis, 2nd edn. Wiley, New York (1989) · Zbl 0718.65001
[2] Barron, A., Birgé, L., Massart, P.: Risk bounds for model selection via penalisation. Probab. Theory Relat. Fields 113, 301-413 (1999) · Zbl 0946.62036 · doi:10.1007/s004400050210
[3] Birgé, L.: The Grenander estimator: a nonasymptotic approach. Ann. Stat. 17(4), 1532-1549 (1989) · Zbl 0703.62042 · doi:10.1214/aos/1176347380
[4] Birgé, L., Massart, P.: Rates of convergence for minimum contrast estimators. Probab. Theory Relat. Fields 97, 113-150 (1993) · Zbl 0805.62037 · doi:10.1007/BF01199316
[5] Bronshtein, E.M.: \[ \epsilon\] ϵ-entropy of convex sets and functions. Sib. Math. J. 17, 393-398 (1976) · Zbl 0354.54026 · doi:10.1007/BF00967858
[6] Cai, T., Low, M.: A framework for estimation of convex functions. Stat. Sin. (to appear). Available at http://www3.stat.sinica.edu.tw/ss_newpaper/SS-13-279_na.pdf (2014) · Zbl 1331.62351
[7] Carolan, C., Dykstra, R.: Asymptotic behavior of the Grenander estimator at density flat regions. Can. J. Stat. 27(3), 557-566 (1999) · Zbl 0949.62035
[8] Cator, E.: Adaptivity and optimality of the monotone least-squares estimator. Bernoulli 17(2), 714-735 (2011) · Zbl 1345.62066 · doi:10.3150/10-BEJ289
[9] Chatterjee, S., Guntuboyina, A., Sen, B.: On risk bounds in isotonic and other shape restricted regression problems (2014, submitted). arXiv:1311.3765 · Zbl 1317.62032
[10] Cule, M.L., Samworth, R.J., Stewart, M.I.: Maximum likelihood estimation of a multi-dimensional log-concave density (with discussion). J. R. Stat. Soci. Ser. B 72, 545-600 (2010) · Zbl 1411.62055 · doi:10.1111/j.1467-9868.2010.00753.x
[11] Dryanov, D.: Kolmogorov entropy for classes of convex functions. Constr. Approx. 30, 137-153 (2009) · Zbl 1177.41022 · doi:10.1007/s00365-009-9053-3
[12] Dümbgen, L., Freitag, S., Jongbloed, G.: Consistency of concave regression with an application to current-status data. Math. Methods Stat. 13(1), 69-81 (2004) · Zbl 1129.62033
[13] Dykstra, R.L.: An algorithm for restricted least squares regression. J. Am. Stat. Assoc. 78(384), 837-842 (1983) · Zbl 0535.62063
[14] Fraser, D.A.S., Massam, H.: A mixed primal-dual bases algorithm for regression under inequality constraints. Application to concave regression. Scand. J. Stat. 16(1), 65-74 (1989) · Zbl 0672.62077
[15] Grenander, U.: On the theory of mortality measurement. II. Skand. Aktuarietidskr. 39, 125-153 (1957) (1956) · Zbl 0077.33715
[16] Groeneboom, P., Jongbloed, G., Wellner, J.A.: Estimation of a convex function: characterizations and asymptotic theory. Ann. Stat. 29(6), 1653-1698 (2001) · Zbl 1043.62027 · doi:10.1214/aos/1015345958
[17] Groeneboom, P., Pyke, R.: Asymptotic normality of statistics based on the convex minorants of empirical distribution functions. Ann. Probab. 11(2), 328-345 (1983) · Zbl 0521.62016 · doi:10.1214/aop/1176993599
[18] Guntuboyina, A., Sen, B.: Covering numbers for convex functions. IEEE Trans. Inf. Theory 59(4), 1957-1965 (2013) · Zbl 1364.52007 · doi:10.1109/TIT.2012.2235172
[19] Hanson, D.L., Pledger, G.: Consistency in concave regression. Ann. Stat. 4(6), 1038-1050 (1976) · Zbl 0341.62034 · doi:10.1214/aos/1176343640
[20] Hildreth, C.: Point estimates of ordinates of concave functions. J. Am. Stat. Assoc. 49, 598-619 (1954) · Zbl 0056.38301 · doi:10.1080/01621459.1954.10483523
[21] Jankowski, H.: Convergence of linear functionals of the Grenander estimator under misspecification. Ann. Stat. 42(2), 625-653 (2014) · Zbl 1302.62045 · doi:10.1214/13-AOS1196
[22] Mammen, E.: Nonparametric regression under qualitative smoothness assumptions. Ann. Stat. 19(2), 741-759 (1991) · Zbl 0737.62039 · doi:10.1214/aos/1176348118
[23] Massart, P.: Concentration Inequalities and Model Selection. Lecture Notes in Mathematics, volume 1896. Springer, Berlin (2007) · Zbl 1170.60006
[24] Pollard, D.: Empirical Processes: Theory and Applications, volume 2 of NSF-CBMS Regional Conference Series in Probability and Statistics. Institute of Mathematical Statistics, Hayward, CA (1990) · Zbl 0741.60001
[25] Rigollet, P., Tsybakov, A.B.: Sparse estimation by exponential weighting. Stat. Sci. 27(4), 558-575 (2012) · Zbl 1331.62351 · doi:10.1214/12-STS393
[26] Seijo, E., Sen, B.: Nonparametric least squares estimation of a multivariate convex regression function. Ann. Stat. 39, 1633-1657 (2011) · Zbl 1220.62044 · doi:10.1214/10-AOS852
[27] Seregin, A., Wellner, J.A.: Nonparametric estimation of multivariate convex-transformed densities. Ann. Stat. 38, 3751-3781 (2010) · Zbl 1204.62058 · doi:10.1214/10-AOS840
[28] Stark, H., Yang, Y.: Vector Space Projections. Wiley, New York (1988)
[29] Van de Geer, S.: Hellinger-consistency of certain nonparametric maximum likelihood estimators. Ann. Stat. 21(1), 14-44 (1993) · Zbl 0779.62033 · doi:10.1214/aos/1176349013
[30] Van de Geer, S.: Applications of Empirical Process Theory. Cambridge University Press, Cambridge (2000) · Zbl 0953.62049
[31] Van der Vaart, A.: Asymptotic Statistics. Cambridge University Press, Cambridge (1998) · Zbl 0910.62001 · doi:10.1017/CBO9780511802256
[32] van der Vaart, A.W., Wellner, J.A.: Weak Convergence and Empirical Process: With Applications to Statistics. Springer, Berlin (1996) · Zbl 0862.60002 · doi:10.1007/978-1-4757-2545-2
[33] Zhang, C.-H.: Risk bounds in isotonic regression. Ann. Stat. 30(2), 528-555 (2002) · Zbl 1012.62045 · doi:10.1214/aos/1021379864
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.