×

Concave regression: value-constrained estimation and likelihood ratio-based inference. (English) Zbl 1416.62212

Summary: We propose a likelihood ratio statistic for forming hypothesis tests and confidence intervals for a nonparametrically estimated univariate regression function, based on the shape restriction of concavity (alternatively, convexity). Dealing with the likelihood ratio statistic requires studying an estimator satisfying a null hypothesis, that is, studying a concave least-squares estimator satisfying a further equality constraint. We study this null hypothesis least-squares estimator (NLSE) here, and use it to study our likelihood ratio statistic. The NLSE is the solution to a convex program, and we find a set of inequality and equality constraints that characterize the solution. We also study a corresponding limiting version of the convex program based on observing a Brownian motion with drift. The solution to the limit problem is a stochastic process. We study the optimality conditions for the solution to the limit problem and find that they match those we derived for the solution to the finite sample problem. This allows us to show the limit stochastic process yields the limit distribution of the (finite sample) NLSE. We conjecture that the likelihood ratio statistic is asymptotically pivotal, meaning that it has a limit distribution with no nuisance parameters to be estimated, which makes it a very effective tool for this difficult inference problem. We provide a partial proof of this conjecture, and we also provide simulation evidence strongly supporting this conjecture.

MSC:

62G08 Nonparametric regression and quantile regression
62G15 Nonparametric tolerance and confidence regions
62G10 Nonparametric hypothesis testing
62G20 Asymptotic properties of nonparametric inference

References:

[1] Aït-Sahalia, Y., Duarte, J.: Nonparametric option pricing under shape restrictions. J. Econom. 116(1-2), 9-47 (2003) · Zbl 1016.62121 · doi:10.1016/S0304-4076(03)00102-7
[2] Allon, G., Beenstock, M., Hackman, S., Passy, U., Shapiro, A.: Nonparametric estimation of concave production technologies by entropic methods. J. Appl. Econom. 22(4), 795-816 (2007) · doi:10.1002/jae.918
[3] Balabdaoui, F., Rufibach, K., Wellner, J.A.: Limit distribution theory for maximum likelihood estimation of a log-concave density. Ann. Stat. 37(3), 1299-1331 (2009) · Zbl 1160.62008 · doi:10.1214/08-AOS609
[4] Banerjee, M.: Likelihood ratio tests for monotone functions. Ph.D. thesis, University of Washington (2000)
[5] Banerjee, M.: Likelihood based inference for monotone response models. Ann. Stat. 35(3), 931-956 (2007). https://doi.org/10.1214/009053606000001578 · Zbl 1133.62328 · doi:10.1214/009053606000001578
[6] Banerjee, M., Wellner, J.A.: Likelihood ratio tests for monotone functions. Ann. Stat. 29(6), 1699-1731 (2001) · Zbl 1043.62037 · doi:10.1214/aos/1015345959
[7] Birke, M., Dette, H.: Estimating a convex function in nonparametric regression. Scand. J. Stat. 34(2), 384-404 (2007) · Zbl 1142.62019 · doi:10.1111/j.1467-9469.2006.00534.x
[8] Bronšteĭn, E.M.: Extremal convex functions. Sibirsk. Mat. Ž. 19(1), 10-18, 236 (1978) · Zbl 0392.52001
[9] Brunk, H.D.: Estimation of isotonic regression. In: Nonparametric techniques in statistical inference (Proceedings of Symposium, Indiana University, Bloomington, IN, 1969), pp. 177-197. Cambridge University Press, London (1970)
[10] Cai, T.T., Low, M.G., Xia, Y.: Adaptive confidence intervals for regression functions under shape constraints. Ann. Stat. 41(2), 722-750 (2013) · Zbl 1267.62066 · doi:10.1214/12-AOS1068
[11] Dent, W.: A note on least squares fitting of functions constrained to be either nonnegative, nondecreasing or convex. Manag. Sci. 20, 130-132 (1973/74) · Zbl 0305.90042
[12] Doss, C.R., Wellner, J.A.: Global rates of convergence of the MLEs of log-concave and \[s\] s-concave densities. Ann. Stat. 44(3), 954-981 (2016) · Zbl 1338.62101 · doi:10.1214/15-AOS1394
[13] Doss, C.R., Wellner, J.A.: Inference for the mode of a log-concave density. Submitted to the Annals of Statistics (2018). arXiv:1611.10348 · Zbl 1439.62098
[14] Doss, C.R., Wellner, J.A.: Log-concave density estimation with symmetry or modal constraints. Submitted to Annals of Statistics (2018). arXiv:1611.10335v2 · Zbl 1422.62137
[15] Dümbgen, L.: Optimal confidence bands for shape-restricted curves. Bernoulli 9(3), 423-449 (2003) · Zbl 1044.62051 · doi:10.3150/bj/1065444812
[16] Dümbgen, L., Rufibach, K.: Maximum likelihood estimation of a log-concave density and its distribution function: basic properties and uniform consistency. Bernoulli 15(1), 40-68 (2009) · Zbl 1200.62030 · doi:10.3150/08-BEJ141
[17] Dykstra, R.L.: An algorithm for restricted least squares regression. J. Am. Stat. Assoc. 78(384), 837-842 (1983) · Zbl 0535.62063 · doi:10.1080/01621459.1983.10477029
[18] Efron, B.: Bootstrap methods: another look at the jackknife. Ann. Stat. 7(1), 1-26 (1979) · Zbl 0406.62024 · doi:10.1214/aos/1176344552
[19] Folland, G.B.: Real Analysis, Pure and Applied Mathematics, 2nd edn. Wiley, New York (1999) · Zbl 0924.28001
[20] Fraser, D.A.S., Massam, H.: A mixed primal-dual bases algorithm for regression under inequality constraints. application to concave regression. Scand. J. Stat. 16(1), 65-74 (1989) · Zbl 0672.62077
[21] Groeneboom, P.; Bernard, P. (ed.), Lectures on inverse problems, 67-164 (1996), Berlin · Zbl 0907.62042
[22] Groeneboom, P., Jongbloed, G.: Nonparametric confidence intervals for monotone functions. Ann. Stat. 43(5), 2019-2054 (2015) · Zbl 1323.62040 · doi:10.1214/15-AOS1335
[23] Groeneboom, P., Jongbloed, G., Wellner, J.A.: A canonical process for estimation of convex functions: the “invelope” of integrated brownian motion \[+t^4\]+t4. Ann. Stat. 29(6), 1620-1652 (2001) · Zbl 1043.62026 · doi:10.1214/aos/1015345957
[24] Groeneboom, P., Jongbloed, G., Wellner, J.A.: Estimation of a convex function: characterizations and asymptotic theory. Ann. Stat. 29(6), 1653-1698 (2001) · Zbl 1043.62027 · doi:10.1214/aos/1015345958
[25] Hall, P.: Effect of bias estimation on coverage accuracy of bootstrap confidence intervals for a probability density. Ann. Stat. 20(2), 675-694 (1992) · Zbl 0748.62028 · doi:10.1214/aos/1176348651
[26] Hannah, L.A., Dunson, D.B.: Multivariate convex regression with adaptive partitioning. J. Mach. Learn. Res. 14, 3261-3294 (2013) · Zbl 1318.62225
[27] Hanson, D.L., Pledger, G.: Consistency in concave regression. Ann. Stat. 4(6), 1038-1050 (1976) · Zbl 0341.62034 · doi:10.1214/aos/1176343640
[28] Hildreth, C.: Point estimates of ordinates of concave functions. J. Am. Stat. Assoc. 49(267), 598-619 (1954) · Zbl 0056.38301 · doi:10.1080/01621459.1954.10483523
[29] Hudson, D.J.: Least-squares fitting of a polynomial constrained to be either non-negative non-decreasing or convex. J. R. Stat. Soc. B 31(1), 113-118 (1969) · Zbl 0177.22703
[30] Johansen, S.: The extremal convex functions. Math. Scand. 34, 61-68 (1974) · Zbl 0286.26002 · doi:10.7146/math.scand.a-11506
[31] Kuosmanen, T.: Representation theorem for convex nonparametric least squares. Econom. J. 11(2), 308-325 (2008) · Zbl 1141.91640 · doi:10.1111/j.1368-423X.2008.00239.x
[32] Lim, E.: Response surface computation via simulation in the presence of convexity. In: Johansson, B., Jain, S., Montoya-Torres, J., Hugan, J., Yücesan, E. (eds.) 2010 Winter Simulation Conference, pp. 1246-1254 (2010)
[33] Lim, E., Glynn, P.W.: Consistency of multidimensional convex regression. Oper. Res. 60(1), 196-208 (2012) · Zbl 1342.62064 · doi:10.1287/opre.1110.1007
[34] Mammen, E.: Nonparametric regression under qualitative smoothness assumptions. Ann. Stat. 19(2), 741-759 (1991) · Zbl 0737.62039 · doi:10.1214/aos/1176348118
[35] Meyer, M.C.: Inference using shape-restricted regression splines. Ann. Appl. Stat. 2(3), 1013-1033 (2008) · Zbl 1149.62033 · doi:10.1214/08-AOAS167
[36] Meyer, M.C.: Constrained penalized splines. Can. J. Stat. 40(1), 190-206 (2012) · Zbl 1236.62033 · doi:10.1002/cjs.10137
[37] Monti, M.M., Grant, S., Osherson, D.N.: A note on concave utility functions. Mind Soc. 4(1), 85-96 (2005) · doi:10.1007/s11299-005-0006-7
[38] Pal, J.K., Woodroofe, M., Meyer, M.: Estimating a Polya frequency function \[{}_22. In\]: Complex Datasets and Inverse Problems, IMS Lecture Notes Monograph Series, vol. 54, pp. 239-249. Institute of Mathematics Statistics, Beachwood (2007)
[39] Pflug, G., Wets, R.J.B.: Shape-restricted nonparametric regression with overall noisy measurements. J. Nonparametr. Stat. 25(2), 323-338 (2013) · Zbl 1297.62092 · doi:10.1080/10485252.2012.754890
[40] Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970) · Zbl 0193.18401 · doi:10.1515/9781400873173
[41] Seijo, E., Sen, B.: Nonparametric least squares estimation of a multivariate convex regression function. Ann. Stat. 39(3), 1633-1657 (2011) · Zbl 1220.62044 · doi:10.1214/10-AOS852
[42] Silverman, B.: On the estimation of a probability density function by the maximum penalized likelihood method. Ann. Stat. 10(3), 795-810 (1982) · Zbl 0492.62034 · doi:10.1214/aos/1176345872
[43] Sinai, Y.G.: Statistics of shocks in solutions of inviscid burgers equation. Commun. Math. Phys. 148(3), 601-621 (1992) · Zbl 0755.60105 · doi:10.1007/BF02096550
[44] Topaloglu, H., Powell, W.B.: An algorithm for approximating piecewise linear concave functions from sample gradients. Oper. Res. Lett. 31(1), 66-76 (2003) · Zbl 1031.90024 · doi:10.1016/S0167-6377(02)00187-6
[45] Toriello, A., Nemhauser, G., Savelsbergh, M.: Decomposing inventory routing problems with approximate value functions. Naval Res. Logist. 57(8), 718-727 (2010) · Zbl 1202.90021 · doi:10.1002/nav.20433
[46] Wang, J.C., Meyer, M.C.: Testing the monotonicity or convexity of a function using regression splines. Can. J. Stat. 39(1), 89-107 (2011) · Zbl 1349.62141 · doi:10.1002/cjs.10094
[47] Wasserman, L.: All of Nonparametric Statistics. Springer Texts in Statistics. Springer, New York (2006) · Zbl 1099.62029
[48] Wu, C.F.: Some algorithms for concave and isotonic regression. Stud. Manag. Sci. 19, 105-116 · Zbl 0535.65097
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.