×
Pan, Zhi-Wei; Xiao, Quan-Wu

Least-square regularized regression with non-iid sampling. (English) Zbl 1176.68163

J. Stat. Plann. Inference 139, No. 10, 3579-3587 (2009).
Summary: We study the least-square regression learning algorithm generated by regularization schemes in reproducing kernel Hilbert spaces. A non-iid setting is considered: the sequence of probability measures for sampling is not identical and the sampling may be dependent. When the sequence of marginal distributions for sampling converges exponentially fast in the dual of a Hölder space and the sampling process satisfies a polynomial strong mixing condition, we derive learning rates for the learning algorithm.

Cited in 10 Documents

MSC:

68T05 Learning and adaptive systems in artificial intelligence
62J02 General nonlinear regression

Keywords:

least-square regularized regression; sampling with non-identical distributions; strong mixing condition; reproducing kernel Hilbert space
Cite Review PDF
Full Text: DOI

References:

[1] Billingsley, P., Convergence of Probability Measures (1968), Wiley: Wiley New York · Zbl 0172.21201
[2] Bousquet, O.; Elisseeff, A., Stability and generation, J. Machine Learning Res., 2, 499-526 (2002) · Zbl 1007.68083
[3] Caponetto, A.; De Vito, E., Optimal rates of regularized least-square algorithm, Found. Comput. Math., 7, 331-368 (2007) · Zbl 1129.68058
[4] Davydov, Y.; Yu, A., The invariance principle for stationary process, Theory Probab. Appl., 14, 487-498 (1970) · Zbl 0219.60030
[5] Dehling, H.; Philipp, W., Almost sure invariance principles for weakly dependent vector-valued random variables, Anal. Probab., 10, 689-701 (1982) · Zbl 0487.60006
[6] De Vito, E.; Caponnetto, A.; Rosasco, L., Model selection for regularized least-square algorithm in learning theory, Found. Comput. Math., 5, 59-85 (2005) · Zbl 1083.68106
[7] Modha, D. S.; Masry, E., Minimum complexity regression estimation with weakly dependent observations, IEEE Trans. Inform. Theory, 42, 2133-2145 (1996) · Zbl 0868.62015
[8] Smale, S.; Zhou, D. X., Shannon sampling and function reconstruction from point values, Bull. Amer. Math. Soc., 41, 279-305 (2004) · Zbl 1107.94007
[9] Smale, S.; Zhou, D. X., Shannon sampling II: connection to learning theory, Appl. Comput. Harmonic Anal., 19, 285-302 (2005) · Zbl 1107.94008
[10] Smale, S.; Zhou, D. X., Learning theory estimates via integral operators and their approximations, Constr. Approx., 26, 153-172 (2007) · Zbl 1127.68088
[11] Smale, S.; Zhou, D. X., Online learning with Markov sampling, Anal. Appl., 7, 87-113 (2009) · Zbl 1170.68022
[12] Steinwart, I.; Hush, D.; Scovel, C., Learning from dependent observations, J. Multivariate Anal., 100, 175-194 (2008) · Zbl 1158.68040
[13] Sun, H. W.; Wu, Q., A note on application of integral operator in learning theory, Appl. Comput. Harmonic Anal., 26, 416-421 (2009) · Zbl 1165.68059
[14] Sun, H.W., Wu, Q., Regularized least square regression with dependent samples. Adv. Comput. Math., to appear. DOI: 10.1007/s10444-008-9099-y.; Sun, H.W., Wu, Q., Regularized least square regression with dependent samples. Adv. Comput. Math., to appear. DOI: 10.1007/s10444-008-9099-y. · Zbl 1191.68535
[15] Wu, Q.; Ying, Y.; Zhou, D. X., Learning rates of least-square regularized regression, Found. Comput. Math., 6, 171-192 (2006) · Zbl 1100.68100
[16] Xu, Y. L.; Chen, D. R., Learning rates of regularized regression for exponentially strongly mixing sequence, J. Statist. Plann. Inference, 138, 2180-2189 (2008) · Zbl 1134.62050
[17] Zhang, T., Leave-one-out bounds for kernel methods, Neural Comput., 15, 1397-1437 (2003) · Zbl 1085.68144
[18] Zhou, D. X., Capacity of reproducing kernel spaces in learning theory, IEEE Trans. Inform. Theory, 49, 1743-1752 (2003) · Zbl 1290.62033
[19] Zhou, D. X., The covering number in learning theory, J. Complexity, 18, 739-767 (2002) · Zbl 1016.68044
[20] Zhou, X.J., Zhou, D.X., Higher order parzen windows and randomized sampling. Adv. Comput. Math., to appear. DOI: 10.1007/s10444-008-9073-8.; Zhou, X.J., Zhou, D.X., Higher order parzen windows and randomized sampling. Adv. Comput. Math., to appear. DOI: 10.1007/s10444-008-9073-8. · Zbl 1183.68514
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.