Approximation by superposition of sigmoidal and radial basis functions. (English) Zbl 0763.41015
Summary: Let \(\sigma:\mathbb{R}\to\mathbb{R}\) be such that for some polynomial \(P\), \(\sigma/P\) is bounded. We consider the linear span of the functions \((\sigma(\lambda(x-t))\); \(\lambda,t\in\mathbb{R}^ s\}\). We prove that unless \(\sigma\) is itself a polynomial, it is possible to uniformly approximate any continuous function on \(\mathbb{R}^ s\) arbitrarily well on every compact subset of \(\mathbb{R}^ s\) by functions in this span. Under more specific conditions on \(\sigma\), we give algorithms to achieve this approximation and obtain Jackson-type theorems to estimate the degree of approximation.
MSC:
| 41A30 | Approximation by other special function classes |
References:
| [1] | Barron, A. R., Universal approximation bounds for superposition of a sigmoidal function, preprint (November 1990) |
| [2] | Caroll, S. M.; Dickinson, S. M., Construction of neural nets using the radon transform, preprint (1990) |
| [3] | Chen, T. P.; Chen, H.; Liu, R. W., A constructive proof of approximation by superposition of sigmoidal functions for neural networks, preprint (1990) |
| [4] | Chui, C. K.; Li, X., Approximation by Ridge Functions and Neural Networks with One Hidden Layer, Texas A & M University CAT Report No. 222 (1990) |
| [5] | Chui, C. K.; Li, X., Realization of Neural Networks with One Hidden Layer, Texas A & M University CAT Report No. 244 (March 1991) |
| [6] | Cybenko, G., Approximation by superposition of sigmoidal functions, Math. Control, Signals Systems, 2, No. 4, 303-314 (1989) · Zbl 0679.94019 |
| [7] | Dahmen, W.; Micchelli, C. A., Some remarks on ridge functions, Approx. Theory Appl., 3, 139-143 (1987) · Zbl 0645.41004 |
| [8] | Dahmen, W.; Micchelli, C. A., Translates of multivariate splines, Linear Algebra Appl., 52/53, 217-234 (1983) · Zbl 0522.41009 |
| [9] | DeVore, R.; Howard, R.; Micchelli, C. A., Optimal nonlinear approximation, Manuscr. Math., 63, 469-478 (1989) · Zbl 0682.41033 |
| [10] | DeVore, R., Degree of approximation, (Lorentz; Chui; Schumaker, Approximation Theory, II (1976), Academic Press: Academic Press New York), 117-161 · Zbl 0343.41017 |
| [11] | Funahashi, K. I., On the approximate realization of continuous mappings by neural networks, Neural Networks, 2, 183-192 (1989) |
| [12] | Hornik, K.; Stinchcombe, M.; White, H., Multilayer feedforward networks are universal approximators, Neural Networks, 2, 359-366 (1989) · Zbl 1383.92015 |
| [13] | Irie, B.; Miyake, S., Capabilities of three layered perceptrons, (IEEE Internat. Conf. Neural Networks, 1 (1988)), 641-648 |
| [14] | Jackson, I. R.H, Convergence properties of radial basis functions, Constr. Approx., 4, 243-264 (1988) · Zbl 0659.41006 |
| [15] | Micchelli, C. A., Interpolation of scattered data: Distance matrices and conditionally positive definite functions, Constr. Approx., 2, 11-22 (1986) · Zbl 0625.41005 |
| [16] | Newman, D. J., Approximation with Rational Functions, (CBMS Regional Conference Series in Mathematics, Vol. 41 (1979), Amer. Math. Soc: Amer. Math. Soc Providence, RI) · Zbl 0417.41004 |
| [17] | Poggio, T.; Girosi, F., Regularization algorithms for learning that are equivalent to multilayer networks, Science, 247, 978-982 (1990) · Zbl 1226.92005 |
| [18] | Powell, M. J.D, The Theory of Radial Basis Function Approximation in 1990, (Numerical Analysis (1990), University of Cambridge), reprint · Zbl 0787.65005 |
| [19] | Rudin, W., Functional Analysis (1973), McGraw-Hill: McGraw-Hill New York · Zbl 0253.46001 |
| [20] | Schumaker, L. L., Spline Functions: Basic Theory (1981), Wiley: Wiley New York · Zbl 0449.41004 |
| [21] | Stinchcombe, M.; White, H., Universal approximation using feedforward network with non-sigmoid hidden layer activation functions, (Proceedings International Joint Conference on Neural Networks (1989), SOS Printing: SOS Printing San Diego), 613-618 |
| [22] | Stinchcombe, M.; White, H., Approximating and learning unknown mappings using multilayer feedforward networks with bounded weights, (IEEE Internat. Conf. Neural Networks, 3 (1990)), III-7-III-16 |
| [23] | Timan, A. F., Theory of Approximation of Functions of a Real Variable (1963), Macmillan Co: Macmillan Co New York · Zbl 0117.29001 |
| [24] | Whitney, H., On functions with \(n\)-th bounded differences, J. Math. Pure Appl., 36, 60-95 (1957) · Zbl 0077.06901 |
| [25] | Zygmund, A., Trigonometric Series (1977), Cambridge Univ. Press: Cambridge Univ. Press Cambridge, U · Zbl 0367.42001 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
