Kernel embedding of measures and low-rank approximation of integral operators. (English) Zbl 07845550
Summary: We describe a natural coisometry from the Hilbert space of all Hilbert-Schmidt operators on a separable reproducing kernel Hilbert space (RKHS) \(\mathcal{H}\) and onto the RKHS \(\mathcal{G}\) associated with the squared-modulus of the reproducing kernel of \(\mathcal{H}\). Through this coisometry, trace-class integral operators defined by general measures and the reproducing kernel of \(\mathcal{H}\) are isometrically represented as potentials in \(\mathcal{G}\), and the quadrature approximation of these operators is equivalent to the approximation of integral functionals on \(\mathcal{G}\). We then discuss the extent to which the approximation of potentials in RKHSs with squared-modulus kernels can be regarded as a differentiable surrogate for the characterisation of low-rank approximation of integral operators.
MSC:
| 46E22 | Hilbert spaces with reproducing kernels (= (proper) functional Hilbert spaces, including de Branges-Rovnyak and other structured spaces) |
| 47B32 | Linear operators in reproducing-kernel Hilbert spaces (including de Branges, de Branges-Rovnyak, and other structured spaces) |
| 47G10 | Integral operators |
| 65F55 | Numerical methods for low-rank matrix approximation; matrix compression |
Keywords:
reproducing kernel Hilbert spaces; integral operators; low-rank approximation; kernel embedding of measures; differentiable relaxationReferences:
| [1] | Aronszajn, N., Theory of reproducing kernels, Trans. Am. Math. Soc., 68, 3, 337-404, 1950 · Zbl 0037.20701 · doi:10.1090/S0002-9947-1950-0051437-7 |
| [2] | Bach, F., On the equivalence between kernel quadrature rules and random feature expansions, J. Mach. Learn. Res., 18, 1, 714-751, 2017 · Zbl 1435.65045 |
| [3] | Conway, J.B.: A Course in Functional Analysis, vol. 96. Springer (2019) |
| [4] | Cucker, F.; Smale, S., On the mathematical foundations of learning, Bull. Am. Math. Soc., 39, 1-49, 2002 · Zbl 0983.68162 · doi:10.1090/S0273-0979-01-00923-5 |
| [5] | Cucker, F., Xuan Zhou, D.: Learning Theory: an Approximation Theory Viewpoint. Cambridge University Press (2007) · Zbl 1274.41001 |
| [6] | Damelin, SB; Hickernell, FJ; Ragozin, DL; Zeng, X., On energy, discrepancy and group invariant measures on measurable subsets of Euclidean space, J. Fourier Anal. Appl., 16, 813-839, 2010 · Zbl 1292.49041 · doi:10.1007/s00041-010-9153-2 |
| [7] | Davidson, K.R.: Nest Algebras: Triangular Forms for Operator Algebras on Hilbert Space, volume 191. Longman Scientific and Technical (1988) · Zbl 0669.47024 |
| [8] | Diestel, J., Uhl, J.J.: Vector Measures. vol. 15. American Mathematical Society (1977) · Zbl 0369.46039 |
| [9] | Drineas, P.; Mahoney, MW, On the Nyström method for approximating a Gram matrix for improved kernel-based learning, J. Mach. Learn. Res., 6, 2153-2175, 2005 · Zbl 1222.68186 |
| [10] | Gauthier, B.; Suykens, J., Optimal quadrature-sparsification for integral operator approximation, SIAM J. Sci. Comput., 40, A3636-A3674, 2018 · Zbl 1490.41018 · doi:10.1137/17M1123614 |
| [11] | Gittens, A.; Mahoney, MW, Revisiting the Nyström method for improved large-scale machine learning, J. Mach. Learn. Res., 17, 1-65, 2016 · Zbl 1367.68223 |
| [12] | Hutchings, M., Gauthier, B.: Energy-based sequential sampling for low-rank PSD-matrix approximation. Preprint https://hal.science/hal-04102664 (2023) |
| [13] | Hutchings, M.; Gauthier, B., Local optimisation of Nyström samples through stochastic gradient descent, Mach. Learn. Optim. Data Sci. LOD, 2022, 123-140, 2023 |
| [14] | Kumar, S.; Mohri, M.; Talwalkar, A., Sampling methods for the Nyström method, J. Mach. Learn. Res., 13, 981-1006, 2012 · Zbl 1283.68292 |
| [15] | Muandet, K.; Fukumizu, K.; Sriperumbudur, B.; Schölkopf, B., Kernel mean embedding of distributions: a review and beyond, Found. Trends Mach. Learn., 10, 1-141, 2017 · Zbl 1380.68336 · doi:10.1561/2200000060 |
| [16] | Müller, A., Integral probability metrics and their generating classes of functions, Adv. Appl. Probab., 29, 2, 429-443, 1997 · Zbl 0890.60011 · doi:10.2307/1428011 |
| [17] | Paulsen, V.I., Raghupathi, M.: An Introduction to the Theory of Reproducing Kernel Hilbert Spaces, Cambridge University Press (2016) · Zbl 1364.46004 |
| [18] | Rasmussen, CE; Williams, CKI, Gaussian Processes for Machine Learning, 2006, Cambridge, MA: MIT press, Cambridge, MA · Zbl 1177.68165 |
| [19] | Rosasco, L.; Belkin, M.; De Vito, E., On learning with integral operators, J. Mach. Learn. Res., 11, 905-934, 2010 · Zbl 1242.62059 |
| [20] | Santin, G.; Schaback, R., Approximation of eigenfunctions in kernel-based spaces, Adv. Comput. Math., 42, 973-993, 2016 · Zbl 1353.65139 · doi:10.1007/s10444-015-9449-5 |
| [21] | Smale, S.; Zhou, D-X, Learning theory estimates via integral operators and their approximations, Constr. Approx., 26, 153-172, 2007 · Zbl 1127.68088 · doi:10.1007/s00365-006-0659-y |
| [22] | Smale, S.; Zhou, D-X, Geometry on probability spaces, Constr. Approx., 30, 311-323, 2009 · Zbl 1187.68270 · doi:10.1007/s00365-009-9070-2 |
| [23] | Sriperumbudur, BK; Fukumizu, K.; Gretton, A.; Schölkopf, B.; Lanckriet, GRG, On the empirical estimation of integral probability metrics, Electron. J. Stat., 6, 1550-1599, 2012 · Zbl 1295.62035 · doi:10.1214/12-EJS722 |
| [24] | Sriperumbudur, BK; Gretton, A.; Fukumizu, K.; Schölkopf, B.; Lanckriet, GRG, Hilbert space embeddings and metrics on probability measures, J. Mach. Learn. Res., 11, 1517-1561, 2010 · Zbl 1242.60005 |
| [25] | Steinwart, I., Christmann, A.: Support Vector Machines. Springer (2008) · Zbl 1203.68171 |
| [26] | Steinwart, I.; Scovel, C., Mercer’s theorem on general domains: on the interaction between measures, kernels, and RKHSs, Constr. Approx., 35, 363-417, 2012 · Zbl 1252.46018 · doi:10.1007/s00365-012-9153-3 |
| [27] | Steinwart, I.; Ziegel, JF, Strictly proper kernel scores and characteristic kernels on compact spaces, Appl. Comput. Harmon. Anal., 51, 510-542, 2021 · Zbl 1462.62774 · doi:10.1016/j.acha.2019.11.005 |
| [28] | Yosida, K.: Functional Analysis. Springer (2012) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
