Some properties of Gaussian reproducing kernel Hilbert spaces and their implications for function approximation and learning theory. (English) Zbl 1204.68157
Summary: We give several properties of the reproducing kernel Hilbert space induced by the Gaussian kernel, along with their implications for recent results in the complexity of the regularized least square algorithm in learning theory.
MSC:
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 68P30 | Coding and information theory (compaction, compression, models of communication, encoding schemes, etc.) (aspects in computer science) |
Keywords:
reproducing kernel Hilbert spaces; Gaussian kernel; eigenvalues; learning theory; regularized least square algorithmReferences:
| [1] | Aronszajn, N.: Theory of reproducing kernels. Trans. Am. Math. Soc. 68, 337–404 (1950) · Zbl 0037.20701 · doi:10.1090/S0002-9947-1950-0051437-7 |
| [2] | Boucheron, S., Bousquet, O., Lugosi, G.: Theory of classification: a survey of recent advances. ESAIM: Prob. Stat. 9, 323–375 (2005) · Zbl 1136.62355 · doi:10.1051/ps:2005018 |
| [3] | Carmeli, C., De Vito, E., Toigo, A.: Vector valued reproducing kernel Hilbert spaces of integrable functions and Mercer theorem. Anal. Appl. 4, 377–408 (2006) · Zbl 1116.46019 · doi:10.1142/S0219530506000838 |
| [4] | Cucker, F., Smale, S.: On the mathematical foundations of learning. Bull. Am. Math. Soc. 39(1), 1–49 (2002) · Zbl 0983.68162 · doi:10.1090/S0273-0979-01-00923-5 |
| [5] | De Vito, E., Caponnetto, A., Rosasco, L.: Model selection for regularized least-squares algorithm in learning theory. Found. Comput. Math. 5(1), 59–85 (2005) · Zbl 1083.68106 · doi:10.1007/s10208-004-0134-1 |
| [6] | Gradshteyn, I.S., Ryzhik, I.M.: Table of Integrals, Series, Products, 6th edn. Academic Press, San Diego (2000) |
| [7] | Mercer, J.: Functions of positive and negative type, and their connection with the theory of integral equations. Philos. Trans. R. Soc. Lond., Ser. A 209, 415–446 (1909) · JFM 40.0408.02 · doi:10.1098/rsta.1909.0016 |
| [8] | Minh, H.Q.: The regularized least square algorithm and the problem of learning halfspaces. Submitted preprint (2007) |
| [9] | Müller, C.: Analysis of Spherical Symmetries in Euclidean Spaces. Applied Mathematical Sciences, vol. 129. Springer, New York (1997) |
| [10] | Niyogi, P., Girosi, F.: Generalization bounds for function approximation from scattered noisy data. Adv. Comput. Math. 10, 51–80 (1999) · Zbl 1053.65506 · doi:10.1023/A:1018966213079 |
| [11] | Poggio, T., Smale, S.: The mathematics of learning: dealing with data. Not. Am. Math. Soc. 50(5), 537–544 (2003) · Zbl 1083.68100 |
| [12] | Schölkopf, B., Smola, A.J.: Learning with Kernels. MIT Press, Cambridge (2002) · Zbl 1019.68094 |
| [13] | Smale, S., Zhou, D.X.: Learning theory estimates via integral operators and their approximations. Constr. Approx. 26(2), 153–172 (2007) · Zbl 1127.68088 · doi:10.1007/s00365-006-0659-y |
| [14] | Steinwart, I.: On the influence of the kernel on the consistency of support vector machines. J. Mach. Learn. Res. 2, 67–93 (2001) · Zbl 1009.68143 · doi:10.1162/153244302760185252 |
| [15] | Steinwart, I., Hush, D., Scovel, C.: An explicit description of the reproducing kernel Hilbert spaces of Gaussian RBF kernels. IEEE Trans. Inf. Theory 52, 4635–4643 (2006) · Zbl 1320.68148 · doi:10.1109/TIT.2006.881713 |
| [16] | Sun, H.W.: Mercer theorem for RKHS on noncompact sets. J. Complex. 21, 337–349 (2005) · Zbl 1094.46021 · doi:10.1016/j.jco.2004.09.002 |
| [17] | Sun, H.W., Zhou, D.X.: Reproducing kernel Hilbert spaces associated with analytic translation-invariant Mercer kernels. J. Fourier Anal. Appl. 14, 89–101 (2008) · Zbl 1153.46017 · doi:10.1007/s00041-007-9003-z |
| [18] | Temlyakov, V.N.: Approximation in learning theory. Constr. Approx. 27, 33–74 (2008) · Zbl 05264756 · doi:10.1007/s00365-006-0655-2 |
| [19] | Tsybakov, A.B.: Optimal aggregation of classifiers in statistical learning. Ann. Stat. 32(1), 135–166 (2004) · Zbl 1105.62353 · doi:10.1214/aos/1079120131 |
| [20] | Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998) · Zbl 0935.62007 |
| [21] | Wahba, G.: Spline Models for Observational Data. CBMS-NSF Regional Conference Series in Applied Mathematics. Society for Industrial and Applied Mathematics, Philadelphia (1990) · Zbl 0813.62001 |
| [22] | Yao, Y.: Early stopping in gradient descent learning. Constr. Approx. 26(2), 289–315 (2007) · Zbl 1125.62035 · doi:10.1007/s00365-006-0663-2 |
| [23] | Ying, Y., Zhou, D.X.: Learnability of Gaussians with flexible variances. J. Mach. Learn. Res. 8, 249–276 (2007) · Zbl 1222.68339 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
