×

Geometric properties of the ridge function manifold. (English) Zbl 1188.41018

The author studies geometrical properties of the ridge function manifold \(\mathcal{R}_n\) consisting of all possible linear combinations of \(n\) functions of the form \(g(a\cdot x)\), where \(a\cdot x\) is the inner product in \({\mathbb R}^d\)and obtains an estimate for the \(\varepsilon \)-entropy numbers in terms of smaller \(\varepsilon \)-covering numbers of the compact class \(G _{n,s }\) formed by the intersection of the class \(\mathcal{R}_n\) with the unit ball \(B\mathcal{P}_s^d\) in the space of polynomials on \({\mathbb R}^d\) of degree \(s\). In particular he shows that for \(n \leq s ^{d - 1}\) the \(\varepsilon \)-entropy number \(H _{\varepsilon }(G _{n,s },L _{q })\) of the class \(G _{n,s }\) in the space \(L _{q }\) is of order \(ns \log 1/\varepsilon \) (modulo a logarithmic factor). Note that the \(\varepsilon \)-entropy number \(H_\varepsilon(B\mathcal{P}_s^d,L_q)\) of the unit ball is of order \(s ^{d }\log 1/\varepsilon \). This result answers a question posed by Allan Pinkus: what is the cardinality of the intersection of the ridge function manifold with the unit ball in the space of polynomials of aa given degree?
The author also obtains an estimate for the pseudo-dimension of the ridge function class \(G _{n,s }\).

MSC:

41A46 Approximation by arbitrary nonlinear expressions; widths and entropy
41A30 Approximation by other special function classes
20-04 Software, source code, etc. for problems pertaining to group theory
54C70 Entropy in general topology

References:

[1] Anthony, M., Bartlett, P.L.: Neural Network Learning: Theoretical Foundations. Cambridge University Press, Cambridge (1999) · Zbl 0968.68126
[2] Karpinski, M., Macintyre, A.J.: Polynomial bounds for VC dimension of sigmoidal neural networks. In: Proc. 27th ACM Synp. Thy. of Computing, pp. 200–208, June 1995 · Zbl 0978.68562
[3] Lin, V., Ya., Pinkus, A.: Fundamentality of ridge functions. J. Approx. Theory 75, 295–311 (1993) · Zbl 0813.41017 · doi:10.1006/jath.1993.1104
[4] Kolmogorov, A.N., Tikhomirov, V.M.: {\(\epsilon\)}-entropy and {\(\epsilon\)}-capacity of sets in functional spaces. In: Shirayev, A.N. (ed.) Selected Works of A.N. Kolmogorov, vol. 3. Kluwer, Dordrecht (1993)
[5] Milnor, J.: On the Betti numbers of real varieties. Proc. Amer. Math. Soc. 15, 275–280 (1964) · Zbl 0123.38302 · doi:10.1090/S0002-9939-1964-0161339-9
[6] Logan, B., Shepp, L.: Optimal reconstruction of functions from its projections. Duke Math. 42, 645–659 (1975) · Zbl 0343.41020 · doi:10.1215/S0012-7094-75-04256-8
[7] Maiorov, V.: On best approximation by ridge functions. J. Approx. Theory 99, 68–94 (1999) · Zbl 0939.41014 · doi:10.1006/jath.1998.3304
[8] Maiorov, V., Pinkus, A.: Lower bounds for approximation by MLP neural networks. Neurocomputing 25, 81–91 (1999) · Zbl 0931.68093 · doi:10.1016/S0925-2312(98)00111-8
[9] Pinkus, A.: Approximation theory of the MLP model in neural networks. Acta Numer. 8, 143–196 (1999) · Zbl 0959.68109 · doi:10.1017/S0962492900002919
[10] Oskolkov, K.I.: Ridge approximation, Chebyshev-Fourier analysis and optimal quadrature formulas. Proc. Steklov Inst. Math. 219, 265–280 (1997) · Zbl 0936.41013
[11] Stein, E.M., Weiss, G.: Introduction to Fourier Analysis on Euclidean Spaces. Princeton University Press, Princeton (1971) · Zbl 0232.42007
[12] Vostrecov, B.A., Kreines, M.A.: Approximation of continuous functions by superpositions of plane waves. Dokl. Akad. Nauk SSSR 140, 1237–1240 (1961) (Soviet Math. Dokl. 2, 1320–1329 (1961))
[13] Vitushkin, A.G.: Estimation of the Complexity of the Tabulation Problem. Fizmatgiz, Moscow (1959) · Zbl 0087.07103
[14] Warren, H.E.: Lower bounds for approximation by nonlinear manifold. Trans. Amer. Math. Soc. 133, 167–178 (1968) · Zbl 0174.35403 · doi:10.1090/S0002-9947-1968-0226281-1
[15] Zygmund, A.: Trigonometric Series, vol. 1. Cambridge University Press, Cambridge (1959) · Zbl 0085.05601
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.