×

Asymptotics and smoothing parameter selection for penalized spline regression with various loss functions. (English) Zbl 1528.62023

Summary: Penalized splines are used in various types of regression analyses, including non-parametric quantile, robust and the usual mean regression. In this paper, we focus on the penalized spline estimator with general convex loss functions. By specifying the loss function, we can obtain the mean estimator, quantile estimator and robust estimator. We will first study the asymptotic properties of penalized splines. Specifically, we will show the asymptotic bias and variance as well as the asymptotic normality of the estimator. Next, we will discuss smoothing parameter selection for the minimization of the mean integrated squares error. The new smoothing parameter can be expressed uniquely using the asymptotic bias and variance of the penalized spline estimator. To validate the new smoothing parameter selection method, we will provide a simulation. The simulation results show that the consistency of the estimator with the proposed smoothing parameter selection method can be confirmed and that the proposed estimator has better behavior than the estimator with generalized approximate cross-validation. A real data example is also addressed.
{© 2016 The Authors. Statistica Neerlandica © 2016 VVS.}

MSC:

62G08 Nonparametric regression and quantile regression
41A15 Spline approximation
62G20 Asymptotic properties of nonparametric inference

Software:

SemiPar
Full Text: DOI

References:

[1] Agarwal, G., and W.Studden (1980), Asymptotic integrated mean square error using least squares and bias minimizing splines, Annals of Statistics8, 1307-1325. · Zbl 0522.62032
[2] Andersen, R. (2008), Modern methods for robust regression, Sage Publications, Thousand Oaks, CA.
[3] Andriyana, Y., I.Gijbels, and A.Verhasselt (2014), Quantile regression in varying coefficient models, Test23, 153-194. · Zbl 1297.62067
[4] Antoniadis, A., I.Gijbels, and A.Verhasselt (2012a), Variable selection in additive models using P‐splines, Technometrics54, 425-438.
[5] Antoniadis, A., I.Gijbels, and A.Verhasselt (2012b), Variable selection in varying coefficient models using P‐splines, Journal of Computational and Graphical Statistics21, 638-661.
[6] Barrow, D. L., and P. W.Smith (1978), Asymptotic properties of best L_2[0,1] approximation by spline with variable knots, The Quarterly of Applied Mathematics36, 293-304. · Zbl 0395.41012
[7] Bollaerts, K., P. H. C.Eilers, and A.Marc (2006), Quantile regression with monotonicity restrictions using P‐splines and L_1‐norm, Statistical Modeling6, 189-207. · Zbl 07257134
[8] Claeskens, G., T.Krivobokova, and J. D.Opsomer (2009), Asymptotic properties of penalized spline estimators, Biometrika96, 529-544. · Zbl 1170.62031
[9] Cox, D. D. (1983), Asymptotics for M‐type smoothing splines, Annals of Statistics11, 530-551. · Zbl 0519.62034
[10] deBoor, C. (2001), A practical guide to splines, Springer‐Verlag, New York. Revised Edition. · Zbl 0987.65015
[11] Eilers, P. H. C., and B. D.Marx (1996), Flexible smoothing with B‐splines and penalties, Statistical Science11, 89-121. · Zbl 0955.62562
[12] Fan, J., T. C.Hu, and Y. K.Truong (1994), Robust nonparametric function estimation, Scandinavian Journal of Statistics21, 433-446. · Zbl 0810.62038
[13] Fahrmeir, L., T.Kneib, and S.Lang (2004), Penalized structured additive regression for space‐time data: a Bayesian perspective, Statistica Sinica14, 731-761. · Zbl 1073.62025
[14] Fahrmeir, L., and T.Kneib (2009), Propriety of posteriors in structured additive regression models: theory and empirical evidence, Journal of Statistical Planning and Inference139, 843-859. · Zbl 1156.62029
[15] Gijbels, I., and A.Verhasselt (2010), P‐splines regression smoothing and difference type of penalty, Statistics and Computing20, 499-511.
[16] Gijbels, I., A.Verhasselt, and I.Vrinssen (2015), Variable selection using P‐splines, Wiley Interdisciplinary Reviews: Computational Statistics7, 1-20. · Zbl 1507.62061
[17] Hall, P., and J. D.Opsomer (2005), Theory for penalized spline regression, Biometrika92, 105-118. · Zbl 1068.62045
[18] Hampel, F. R., E. M.Ronchetti, P. J.Rousseeuw, and W. A.Stahel (2005), Robust statistics: the approach based on influence functions, (Wiley Series in Probability and Statistics), Wiley‐Interscience, New York.
[19] Hao, L., and D. Q.Naiman (2007), Quantile regression, Sage Publications, Inc., Thousand Oaks.
[20] Hastie, T., R.Tibshirani, and J.Friedman (2009), The elements of statistical learning, Springer‐Verlag, New York. · Zbl 1273.62005
[21] He, X., and P.Shi (1994), Convergence rate of B‐spline estimators of nonparametric conditional quantile functions, Journal of Nonparametric Statistics3, 299-308. · Zbl 1383.62111
[22] Heinzl, F., T.Kneib, and L.Fahrmeir (2012), Additive mixed models with Dirichlet process mixture and P‐spline priors, Advances in Statistical Analysis96, 47-68. · Zbl 1443.62098
[23] Huber, P. J. (1979). Robust smoothing, in: Launer, R. L. (ed.), and G. N.Wilkinson (ed.) (eds), Robustness in Statistics, New York; London, Academic Press, 33-48.
[24] Huber, P., and E. M.Ronchetti (2009), Robust statistics Second edition, Wiley, Hoboken, NJ. · Zbl 1276.62022
[25] Kauermann, G., T.Krivobokova, and L.Fahrmeir (2009), Some asymptotic results on generalized penalized spline smoothing, Journal of the Royal Statistical Society: Series B71, 487-503. · Zbl 1248.62055
[26] Kauermann, G. (2005), Penalised Spline Fitting in Multivariable Survival Models with Varying Coefficients, Computational Statistics and Data Analysis49, 169-186. · Zbl 1429.62141
[27] Koenker, R. (2005), Quantile regression, Cambridge University Press, Cambridge. · Zbl 1111.62037
[28] Koenker, R., and B. J.Park (1996), An interior point algorithm for nonlinear quantile regression, Journal of Econometrics71, 265-283. · Zbl 0855.62030
[29] Krivobokova, T. (2013), Smoothing parameter selection in two frameworks for penalized splines, Journal of the Royal Statistical Society: Series B75, 725-741. · Zbl 1411.62102
[30] Lee, T. C. M., and H.Oh (2007), Robust penalized regression spline fitting with application to additive mixed modeling, Computational Statistics22, 159-171. · Zbl 1194.62046
[31] Li, Y., and D.Ruppert (2008), On the asymptotics of penalized splines, Biometrika95, 415-436. · Zbl 1437.62540
[32] Lin, X., and D.Zhang (1999), Inference in generalized additive mixed models by using smoothing splines, Journal of the Royal Statistical Society: Series B61, 381-400. · Zbl 0915.62062
[33] Meyer, M. C. (2012), Constrained penalized splines, The Canadian Journal of Statistics40, 190-206. · Zbl 1236.62033
[34] O’Sullivan, F. (1986), A statistical perspective on ill‐posed inverse problems (with discussion), Statistical Science1, 505-527. · Zbl 0625.62110
[35] Pollard, D. (1991), Asymptotics for least absolute deviation regression estimators, Econometric Theory7, 186-199.
[36] Pratesi, M., M. G.Ranalli, and N.Salvati (2009), Nonparametric M‐quantile regression using penalised splines, Journal of Nonparametric Statistics21, 287-304. · Zbl 1158.62033
[37] Reiss, P. T., and L.Huang (2012), Smoothness selection for penalized quantile regression splines, The International Journal of Biostatistics8.1.
[38] Reiss, P. T., and R. T.Ogden (2009), Smoothing parameter selection for a class of semiparametric linear models, Journal of the Royal Statistical Society: Series B71, 503-523. · Zbl 1248.62057
[39] Ruppert, D. (2002), Selecting the number of knots for penalized splines, JCGS11, 735-757.
[40] Ruppert, D., M. P.Wand, and R. J.Carroll (2003), Semiparametric regression, Cambridge University Press, Cambridge. · Zbl 1038.62042
[41] Ruppert, D., M. P.Wand, and R. J.Carroll (2009), Semiparametric regression during 2003‐2007, Electronic Journal of Statistics3, 1193-1256. · Zbl 1326.62094
[42] Silverman, B. W. (1985), Some aspects of the spline smoothing approach to nonparametric regression curve fitting, Journal of the Royal Statistical Society: Series B47, 1-?52. · Zbl 0606.62038
[43] Tharmaratnam, K., G.Claeskens, CCroux, and M.Salibian‐Barrera (2010), S‐Estimation for penalized regression splines, Journal of Computational and Graphical Statistics19(3), 609-625.
[44] Wand, M. P. (2003), Smoothing and mixed models, Computational Statistics18, 223-249. · Zbl 1050.62049
[45] Wand, M., and J.Ormerod (2008), On semiparametric regression with O’Sullivan penalised splines, Australian & New Zealand Journal of Statistics50, 179-198. · Zbl 1146.62030
[46] Wang, X., J.Shen, and D.Ruppert (2011), On the asymptotics of penalized spline smoothing, Electronic Journal of Statistics5, 1-17. · Zbl 1274.65012
[47] Wood, S. N. (2000), Modeling and smoothing parameter estimation with multiple quadratic penalties, Journal of the Royal Statistical Society: Series B62, 413-428.
[48] Wood, S. N. (2004), Stable and efficient multiple smoothing parameter estimation for generalized additive models, Journal of the American Statistical Association99, 673-686. · Zbl 1117.62445
[49] Wood, S. N. (2008), Fast stable direct fitting and smoothness selection for generalized additive models, Journal of the Royal Statistical Society: Series B70, 495-518. · Zbl 05563356
[50] Xiao, L., Y.Li, and D.Ruppert (2013), Fast bivariate P‐splines: the sandwich smoother, J. R. Statist. Soc. B75, 577-599. · Zbl 1411.62109
[51] Yoshida, T. (2013a), Asymptotics for penalized spline estimators in quantile regression, Communications in Statistics‐Theory and Methods, DOI: 10.1080/03610926.2013.765477. · Zbl 07710569
[52] Yoshida, T. (2013b), Direct determination of smoothing parameter for penalized spline regression, Journal of Probability and Statistics, Article ID: 203469. · Zbl 1307.62119
[53] Yoshida, T., and K.Naito (2014), Asymptotics for penalized splines in generalized additive models, Journal of Nonparametric Statistics47, 269-289. · Zbl 1359.62307
[54] Zhou, S., X.Shen, and D. A.Wolfe (1998), Local asymptotics for regression splines and confidence regions, Annals of Statistics26, 1760-1782. · Zbl 0929.62052
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.