Non-asymptotic error bound for optimal prediction of function-on-function regression by RKHS approach. (English) Zbl 1492.60295
Summary: In this paper, we study and analyze the regularized least squares for function-on-function regression model. In our model, both the predictors (input data) and responses (output data) are multivariate functions (with \(d\) variables and \(\tilde{d}\) variables respectively), and the model coefficient lies in a reproducing kernel Hilbert space (RKHS). We show under mild condition on the reproducing kernel and input data statistics that the convergence rate of excess prediction risk by the regularized least squares is minimax optimal. Numerical examples based on medical image analysis and atmospheric point spread function estimation are considered and tested, and the results demonstrate that the performance of the proposed model is comparable with that of other testing methods.
MSC:
| 60K35 | Interacting random processes; statistical mechanics type models; percolation theory |
| 62J05 | Linear regression; mixed models |
Keywords:
regularized least squares; function-on-function regression; non-asymptotic error bound; reproducing kernel Hilbert space; integral operatorSoftware:
fda (R)References:
| [1] | Berisha, S.; Nagy, J. G.; Plemmons, R. J., Estimation of atmospheric PSF parameters for hyperspectral imaging, Numer. Linear Algebra Appl., 22, 795-813 (2015) · Zbl 1374.94044 · doi:10.1002/nla.1986 |
| [2] | Cai, T.; Hall, P., Prediction in functional linear regression, Ann. Statist., 34, 2159-2179 (2006) · Zbl 1106.62036 |
| [3] | Cai, T.; Yuan, M., Minimax and adaptive prediction for functional linear regression, J. Amer. Statist. Assoc., 107, 1201-1216 (2012) · Zbl 1443.62196 · doi:10.1080/01621459.2012.716337 |
| [4] | Caponnetto, A.; DeVito, E., Optimal rates for the regularized least-squares algorithm, Found. Comput. Math., 7, 331-368 (2007) · Zbl 1129.68058 · doi:10.1007/s10208-006-0196-8 |
| [5] | Crambes, C.; Mas, A., Asymptotics of prediction in functional linear regression with functional outputs, Bernoulli, 19, 2627-2651 (2013) · Zbl 1280.62084 · doi:10.3150/12-BEJ469 |
| [6] | Ferraty, F.; Van Keilegom, I.; Vieu, P., Regression when both response and predictor are functions, J. Mutivariate Anal., 109, 10-28 (2012) · Zbl 1241.62054 · doi:10.1016/j.jmva.2012.02.008 |
| [7] | Ferraty, F.; Vieu, P., Nonparametric Functional Data Analysis: Methods, Theory, Applications and Implementations (2006), New York: Springer, New York · Zbl 1119.62046 |
| [8] | Gonzalez, R. C.; Woods, R. E., Digital Image Processing, Addison-Wesley publishing company, Chapter, 4, 254-256 (1993) · Zbl 0816.68137 |
| [9] | Guntuboyina, A.; Sen, B., Nonparametric shape-restricted regression, Statist. Sci., 33, 568-594 (2018) · Zbl 1407.62135 · doi:10.1214/18-STS665 |
| [10] | Hall, P.; Horowitz, G. L., Methodology and convergence rates for functional linear regression, Ann. Statist., 35, 70-91 (2007) · Zbl 1114.62048 |
| [11] | Hsing, T.; Eubank, R., Theoretical Foundations of Functional Data Analysis, with an Introduction to Linear Operators (2015), Chichester: John Wiley & Sons, Chichester · Zbl 1338.62009 · doi:10.1002/9781118762547 |
| [12] | Hu, Y.; He, X.; Tao, J., Modeling and prediction of childrens growth data via functional principal component analysis, Sci. China Ser. A, 52, 1342-1350 (2009) · Zbl 1176.62061 · doi:10.1007/s11425-009-0088-5 |
| [13] | Ivanescu, A. E.; Staicu, A. M.; Scheipl, F., Penalized function-on-function regression, Comput. Statist., 30, 539-568 (2015) · Zbl 1317.65037 · doi:10.1007/s00180-014-0548-4 |
| [14] | Kokoszka, P.; Reimherr, M., Introduction to Functional Data Analysis (2017), Boca Raton: CRC Press, Boca Raton · Zbl 1411.62004 · doi:10.1201/9781315117416 |
| [15] | Li, X.; Xu, D.; Zhou, H., Tucker tensor regression and neuroimaging analysis, Statistics in Biosciences, 10, 520-545 (2018) · doi:10.1007/s12561-018-9215-6 |
| [16] | Li, Y.; Tsing, T., On rates of convergence in functional linear regression, J. Multivariate Anal., 98, 1782-1804 (2007) · Zbl 1130.62035 · doi:10.1016/j.jmva.2006.10.004 |
| [17] | Lian, H., Minimax prediction for functional linear regression with functional responses in reproducing kernel Hilbert spaces, J. Multivariate Anal., 140, 395-402 (2015) · Zbl 1327.62216 · doi:10.1016/j.jmva.2015.06.005 |
| [18] | Manolakis, D.; Marden, D.; Shaw, G. A., Hyperspectral image processing for automatic target detection applications, Lincoln Laboratory Journal, 14, 79-116 (2003) |
| [19] | Moffat, A. F J., A theoretical investigation of focal stellar images in the photographic emulsion and application to photographic photometry, Astronomy and Astrophysics, 3, 455-461 (1969) |
| [20] | Papp, E.; Cudahy, T., Hyperspectral remote sensing, Geophysical and Remote Sensing Methods for Regolith Exploration, 144, 13-21 (2002) |
| [21] | Paulsen, V.; Raghupathi, M., An Introduction to the Theory of Reproducing Kernel Hilbert Spaces (2016), Cambridge: Cambridge University Press, Cambridge · Zbl 1364.46004 · doi:10.1017/CBO9781316219232 |
| [22] | Pinelis, I., Optimum bounds for the distributions of martingales in Banach spaces, Ann. Probab., 22, 679-1706 (1994) · Zbl 0836.60015 · doi:10.1214/aop/1176988477 |
| [23] | Preda, C., Regression models for functional data by reproducing kernel Hilbert spaces methods, J. Statist. Plann. Inference, 137, 829-840 (2007) · Zbl 1104.62043 · doi:10.1016/j.jspi.2006.06.011 |
| [24] | Ramsay, J. O.; Silverman, B. W., Applied Functional Data Analysis: Methods and Case Studies (2002), New York: Springer, New York · Zbl 1011.62002 · doi:10.1007/b98886 |
| [25] | Ramsay, J. O.; Silverman, B. W., Functional Data Analysis (2005), New York: Springer, New York · Zbl 1079.62006 · doi:10.1007/b98888 |
| [26] | Reiß, M.; Wahl, M., Non-asymptotic upper bounds for the reconstruction error of PCA, Ann. Statist., 48, 1098-1123 (2020) · Zbl 1450.62070 |
| [27] | Rice, J. A.; Silverman, B. W., Estimating mean and covariance structure nonparametrically when the data are curves, J. R. Statist. Soc. B, 53, 233-243 (1991) · Zbl 0800.62214 |
| [28] | Serre, D., Villeneuve, E., Carfantan, H., et al.: Modeling the spatial PSF at the VLT focal plane for MUSE WFM data analysis purpose. (In: SPIE Astronomical Telescopes and Instrumentation: Observational Frontiers of Astronomy for the New Decade), 773649-773649, 2010 |
| [29] | Sheng, B.; Ye, P.; Wang, J., Learning rates for least square regressions with coefficient regularization, Acta Math. Sin., Engl. Ser., 28, 2205-2212 (2012) · Zbl 1258.68071 · doi:10.1007/s10114-012-0607-0 |
| [30] | Smith, R. B.: Introduction to hyperspectral imaging. Tmips, MicroImages Tutorial Web Site: https://www.microimages.com/docamentation/Tutorials/hyprspec.pdf |
| [31] | Špiclin, Ž., Pernuš, F., Likar, B.: Correction of axial optical aberrations in hyperspectral imaging systems. In: Proc. of SPIE, Vol., 7891, 78910S1, 2011 · Zbl 1373.94013 |
| [32] | Steinwart, I., Support vector machines are universally consistent, J. Complexity, 18, 768-791 (2002) · Zbl 1030.68074 · doi:10.1006/jcom.2002.0642 |
| [33] | Steinwart, I., On the influence of the kernel on the consistency of support vector machines, J. Mach. Learning Res., 2, 67-93 (2001) · Zbl 1009.68143 |
| [34] | Sun, X.; Du, P.; Wang, X., Optimal penalized function-on-function regression under a reproducing kernel Hilbert space framework, J. Amer. Statist. Assoc., 113, 1601-1611 (2018) · Zbl 1409.62137 · doi:10.1080/01621459.2017.1356320 |
| [35] | Tong, H.; Ng, M., Analysis of regularized least squares for functional linear regression model, J. Complexity, 49, 85-94 (2018) · Zbl 1402.62158 · doi:10.1016/j.jco.2018.08.001 |
| [36] | Villeneuve, E., Carfantan, H., Serre, D.: PSF estimation of hyperspectral data acquisition system for ground- based astrophysical observations. In: IEEE 3rd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS), 2011 |
| [37] | Wahba, G., Spline Models for Observational Data (1990), Philadelphia: SIAM, Philadelphia · Zbl 0813.62001 · doi:10.1137/1.9781611970128 |
| [38] | Yao, F.; Müller, H. G.; Wang, J. L., Functional linear regression analysis for longitudinal data, Ann. Statist., 33, 2873-2903 (2005) · Zbl 1084.62096 · doi:10.1214/009053605000000660 |
| [39] | Yuan, M.; Cai, T., A reproducing kernel Hilbert space approach to functional linear regression, Ann. Statist., 38, 3412-3444 (2010) · Zbl 1204.62074 · doi:10.1214/09-AOS772 |
| [40] | Zhang, T., Learning bounds for kernel regression using effective data dimensionality, Neural Comput., 17, 2077-2098 (2004) · Zbl 1080.68044 · doi:10.1162/0899766054323008 |
| [41] | Zhao, H.; Li, Y.; Zhao, Y., Empirical likelihood inference for functional coefficient ARCH-M model, Acta Math. Sin., Engl. Ser., 35, 270-296 (2019) · Zbl 1415.62069 · doi:10.1007/s10114-018-8083-9 |
| [42] | Zhou, H.; Li, L.; Zhu, H., Tensor regression with applications in neuroimaging data analysis, J. Amer. Statist. Assoc., 108, 540-552 (2013) · Zbl 06195959 · doi:10.1080/01621459.2013.776499 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
