Truncation level selection in nonparametric regression using Padé approximation. (English) Zbl 1489.62120
Summary: This paper introduces a Padé-type approximation for an unknown regression function in a nonparametric regression model. This newly introduced approximation provides a linear model with multi-collinearities and errors in all its variables. To deal with these issues, we used the truncated total least squares (TTLS) method. The efficient implementation of a Padé-type method using TTLS depends on choosing a truncation level. To provide an optimum truncation level for this method, we update the conventional parameter selection methods, including the generalized cross validation (GCV), improved version of the Akaike information criterion (AICc), restricted maximum likelihood (REML), Bayesian information criterion (BIC), and Mallows’ \(C_p\) criterion. The primary aim of this study is to compare the performances of these level selection methods. A Monte Carlo simulation and a real data example are performed to illustrate the ideas in the paper. The results confirm that the GCV and AICc slightly outperform the other methods, especially when sample sizes are small and large, respectively.
MSC:
| 62G08 | Nonparametric regression and quantile regression |
| 62J05 | Linear regression; mixed models |
| 65D10 | Numerical smoothing, curve fitting |
References:
| [1] | Baker, G., Essentials of padé approximants (1975), New York: Academic Press, New York · Zbl 0315.41014 |
| [2] | Baker, G. A. Jr.; Graves-Morris, P., Padé approximations (1996), Cambridge: Cambridge University Press, Cambridge · Zbl 0923.41001 |
| [3] | Breiman, L., Bagging predictors, Machine Learning, 24, 2, 123-40 (1996) · Zbl 0858.68080 · doi:10.1007/BF00058655 |
| [4] | Brezinski, C., Padé-type approximation and general orthogonal polynomials (1980), Basel: Birkhäuser, Basel · Zbl 0418.41012 |
| [5] | Brezinski, C.; Rodriguez, G.; Seatzu, S., Error estimates for the regularization of least squares problems, Numerical Algorithms, 51, 1, 61-76 (2009) · Zbl 1166.65331 · doi:10.1007/s11075-008-9243-2 |
| [6] | Buja, A., Hastie, T., and Tibshirani, R.. 1987. Linear smoothers and additive models, Technical report. Vol. 106, University of Washington. · Zbl 0689.62029 |
| [7] | Chen, C. S.; Huang, H. C., An improved Cp criterion for spline smoothing, Journal of Statistical Planning and Inference, 141, 1, 445-52 (2011) · Zbl 1203.62062 · doi:10.1016/j.jspi.2010.06.017 |
| [8] | Cheney, E. W., Introduction to approximation theory (1996), Vermont: Chelsea Publishing Company, Vermont · Zbl 0161.25202 |
| [9] | Craven, P.; Wahba, G., Smoothing noisy data with spline functions, Numerische Mathematik, 31, 4, 377-403 (1978) · Zbl 0377.65007 · doi:10.1007/BF01404567 |
| [10] | Eckart, C.; Young, G., The approximation of one matrix by another of lower rank, Psychometrika, 1, 3, 211-8 (1936) · JFM 62.1075.02 · doi:10.1007/BF02288367 |
| [11] | Eubank, R., Nonparametric regression and spline smoothing (1999), New York: Marcel Dekker, New York · Zbl 0936.62044 |
| [12] | Fierro, R. D.; Golub, G. H.; Hansen, P. C.; O’Leary, D. P., Regularization by truncated total least squares, SIAM Journal on Scientific Computing, 18, 4, 1223-41 (1997) · Zbl 0891.65040 · doi:10.1137/S1064827594263837 |
| [13] | Golub, G. H.; Van Loan, C. F., An analysis of the total least squares problem, SIAM Journal on Numerical Analysis, 17, 6, 883-93 (1980) · Zbl 0468.65011 · doi:10.1137/0717073 |
| [14] | Golub, G. H.; Van Loan, C. F., Matrix computations (2013), Baltimore: The Johns Hopkins University Press, Baltimore · Zbl 1268.65037 |
| [15] | Green, P. J.; Silverman, B. W., Nonparametric regression and generalized linear model (1994), Boca Raton: Chapman & Hall, Boca Raton · Zbl 0832.62032 |
| [16] | Hastie, T.; Tibshirani, R.; Friedman, J., The elements of statistical learning. Data mining, inference and prediction (2001), Berlin: Springer, Berlin · Zbl 0973.62007 |
| [17] | Henderson, D. J.; Parmeter, C. F., Applied nonparametric econometrics (2015), New York: Cambridge University Press, New York · Zbl 1305.62004 |
| [18] | Hurvich, C. M.; Simonoff, J. S.; Tsai, C.-L., Smoothing parameter selection in nonparametric regression using an improved akaike information criterion, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 60, 2, 271-93 (1998) · Zbl 0909.62039 · doi:10.1111/1467-9868.00125 |
| [19] | Lee, T. C. M., Improved smoothing spline regression by combining estimates of different smoothness, Statistics & Probability Letters, 67, 133-40 (2004) · Zbl 1058.62035 · doi:10.1016/j.spl.2004.01.003 |
| [20] | Li, Q.; Racine, S., Nonparametric econometrics, Theory and practice (2007), Princeton NJ: Princeton University Press, Princeton NJ · Zbl 1183.62200 |
| [21] | Mallows, C., Some comments on Cp, Technometrics, 15, 661-75 (1973) · Zbl 0269.62061 · doi:10.1080/00401706.1973.10489103 |
| [22] | Mirsky, L., Symmetric gauge functions and unitarily invariant norms, The Quarterly Journal of Mathematics, 11, 1, 50-9 (1960) · Zbl 0105.01101 · doi:10.1093/qmath/11.1.50 |
| [23] | Nadaraya, E. A., On estimating regression, Theory of Probability and Its Applications, 10, 1, 186-90 (1964) · Zbl 0136.40902 |
| [24] | Petrushev, P. P.; Popov, V. A., Rational approximation of real functions (1987), New York: Cambridge University Press, New York · Zbl 0644.41010 |
| [25] | Ruppert, D., Selecting the number of knots for penalized splines, Journal of Computational and Graphical Statistics, 11, 4, 735-57 (2002) · doi:10.1198/106186002853 |
| [26] | Reis, P. T.; Ogden, R. T., Smoothing parameter selection for a class of semiparametric linear models, Journal of the Royal Statistical Society: Series B, 71, 505-23 (2009) · Zbl 1248.62057 |
| [27] | Schimek, G. M., Smoothing and regression: approaches, computation, and application, 19-39 (2000), New York: John Wiley & Sons, Inc, New York · Zbl 0946.00011 |
| [28] | Schwarz, G., Estimating the dimension of a model, The Annals of Statistics, 6, 2, 461-4 (1978) · Zbl 0379.62005 · doi:10.1214/aos/1176344136 |
| [29] | Sima, D. M.; Huffel, S., Level choice in truncated total least squares, Computational Statistics and Data Analysis, 52, 1208-22 (2007) · Zbl 1452.62125 · doi:10.1016/j.csda.2007.05.015 |
| [30] | Tikhonov, A. N., Solution of incorrectly formulated problems and the regularization method, Soviet Mathematics Doklady, 4, 1035-8 (1963) · Zbl 0141.11001 |
| [31] | Van Huffel, S.; Vandewalle, J., The total least squares problem, computational aspects and analysis (1991), SIAM · Zbl 0789.62054 |
| [32] | Wahba, G., Bayesian confidence intervals for the cross-validated smoothing spline, Journal of the Royal Statistical Society, Series B, 45, 1, 133-50 (1983) · Zbl 0538.65006 · doi:10.1111/j.2517-6161.1983.tb01239.x |
| [33] | Watson, G. S., Smooth regression analysis, Sankhya, Series A, 26, 359-72 (1964) · Zbl 0137.13002 |
| [34] | Wuytack, L., Lectures notes in mathematics, 765, Padé approximation and its applications (1979), Berlin: Springer, Berlin · Zbl 0424.00010 |
| [35] | Zhang, D.; Cherkaev, E., Reconstruction of spectral function from effective permittivity of a composite material using rational function approximations, Journal of Computational Physics, 228, 15, 5390-409 (2009) · Zbl 1280.78005 · doi:10.1016/j.jcp.2009.04.014 |
| [36] | Zhang, D.; Cherkaev, E., Pade approximations for identification of air bubble volume from temperature or frequency dependent permittivity of a two-component mixture, Inverse Problems in Science and Engineering, 16, 4, 425-45 (2008) · Zbl 1258.76166 · doi:10.1080/17415970701529213 |
| [37] | Zhang, D.; Lamoureux, M. P.; Margrave, G. F.; Cherkaev, E., Rational approximation for estimation of quality Q factor and phase velocity in linear, viscoelastic, isotropic media, Computational Geosciences, 15, 117-33 (2011) · Zbl 1209.86008 · doi:10.1007/s10596-010-9201-7 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
