Approximation schemes for functional optimization problems. (English) Zbl 1176.90658
Approximation schemes and suboptimal solutions for functional optimization problems with admissible solutions dependent on a large number of variables are investigated on the basis of the analysis for a sufficiently large number \(n\) of basis functions in the paper of V. Kurkova and M. Sanguneti [SIAM J. Optim. 15, 461–487 (2005; Zbl 1074.49008)] in which the critical term is of the form \(n^{- {1 \over 2}}\), multiplied by a certain norm of the optimal solution, called “variation norm”. In the present paper, the authors show that the upper bounds are of the same order \(n^{-\frac12}\) and the error of the approximate optimization can be obtained for various sets of admissible solutions defined in terms of variation norms with respect to different bases, if one suitably “matches” the approximation with the set of admissible solutions. Variation norms are compared for some variable-basis approximation schemes and are estimated in terms of spectral norms. The comparison is given for classical \(L_p\) and Sobolev’s spaces with sets of admissible functions having finite variation norms with respect to some bases.
Reviewer: Tran Nhu Pham (Hanoi)
MSC:
| 90C59 | Approximation methods and heuristics in mathematical programming |
| 49M15 | Newton-type methods |
| 90C48 | Programming in abstract spaces |
Keywords:
complexity of admissible solutions; upper bounds on accuracy; curse of dimensionality; Ritz method; extended Ritz methodCitations:
Zbl 1074.49008References:
| [1] | Gelfand, I.M., Fomin, S.V.: Calculus of Variations. Prentice-Hall, Englewood Cliffs (1963) · Zbl 0127.05402 |
| [2] | Kurková, V., Sanguineti, M.: Error estimates for approximate optimization by the extended Ritz method. SIAM J. Optim. 15, 461–487 (2005) · Zbl 1074.49008 · doi:10.1137/S1052623403426507 |
| [3] | Zoppoli, R., Sanguineti, M., Parisini, T.: Approximating networks and extended Ritz method for the solution of functional optimization problems. J. Optim. Theory Appl. 112, 403–440 (2002) · Zbl 0992.65072 · doi:10.1023/A:1013662124879 |
| [4] | Kurková, V., Sanguineti, M.: Comparison of worst case errors in linear and neural network approximation. IEEE Trans. Inf. Theory 48, 264–275 (2002) · Zbl 1059.62589 · doi:10.1109/18.971754 |
| [5] | Bellman, R.: Dynamic Programming. Princeton University Press, Princeton (1957) · Zbl 0077.13605 |
| [6] | Barron, A.R.: Universal approximation bounds for superpositions of a sigmoidal function. IEEE Trans. Inf. Theory 39, 930–945 (1993) · Zbl 0818.68126 · doi:10.1109/18.256500 |
| [7] | Breiman, L.: Hinging hyperplanes for regression, classification, and function approximation. IEEE Trans. Inf. Theory 39, 999–1013 (1993) · Zbl 0793.62031 · doi:10.1109/18.256506 |
| [8] | Girosi, F.: Regularization theory, radial basis functions and networks. In: Cherkassky, V., Friedman, J.H., Wechsler, H. (eds.) From Statistics to Neural Networks. Theory and Pattern Recognition Applications, Subseries F, pp. 166–187. Springer, Berlin (1994) · Zbl 0837.68093 |
| [9] | Girosi, F., Anzellotti, G.: Rates of convergence for radial basis functions and neural networks. In: Mammone, R.J. (ed.) Artificial Neural Networks for Speech and Vision, pp. 97–113. Chapman & Hall, London (1993) |
| [10] | Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural networks architectures. Neural Comput. 7, 219–269 (1995) · doi:10.1162/neco.1995.7.2.219 |
| [11] | Jones, L.K.: A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training. Ann. Stat. 20, 608–613 (1992) · Zbl 0746.62060 · doi:10.1214/aos/1176348546 |
| [12] | Kainen, P.C., Kurková, V., Sanguineti, M.: Minimization of error functionals over variable-basis functions. SIAM J. Optim. 14, 732–742 (2003) · Zbl 1061.49020 · doi:10.1137/S1052623402401233 |
| [13] | Kainen, P.C., Kurková, V., Sanguineti, M.: Complexity of Gaussian radial basis networks approximating smooth functions. J. Complex. doi: 10.1016/j.jco.2008.08.001 (2008) · Zbl 1162.65006 |
| [14] | Kurková, V.: Dimension-independent rates of approximation by neural networks. In: Warwick, K., Kárný, M. (eds.) Computer-Intensive Methods in Control and Signal Processing: The Curse of Dimensionality, pp. 261–270. Birkhäuser, Boston (1997) |
| [15] | Kurková, V., Sanguineti, M.: Bounds on rates of variable-basis and neural-network approximation. IEEE Trans. Inf. Theory 47, 2659–2665 (2001) · Zbl 1008.41012 · doi:10.1109/18.945285 |
| [16] | Kurková, V., Sanguineti, M.: Geometric upper bounds on rates of variable-basis approximation, IEEE Trans. Inf. Theory (to appear) · Zbl 1080.49020 |
| [17] | Mhaskar, H.N.: Neural networks for optimal approximation of smooth and analytic functions. Neural Comput. 8, 164–177 (1996) · doi:10.1162/neco.1996.8.1.164 |
| [18] | Alessandri, A., Parisini, T., Sanguineti, M., Zoppoli, R.: Neural strategies for nonlinear optimal filtering. In: Proc. of IEEE Int. Conf. on Syst. Eng., Kobe, Japan, pp. 44–49 (1992) |
| [19] | Zoppoli, R., Parisini, T.: Learning techniques and neural networks for the solution of N-stage nonlinear nonquadratic optimal control problems. In: Isidori, A., Tarn, T.J. (eds.) Systems, Models and Feedback: Theory and Applications, pp. 193–210. Birkhäuser, Boston (1992) · Zbl 0776.49024 |
| [20] | Zoppoli, R., Parisini, T., Baglietto, M., Sanguineti, M.: Neural Approximations for Optimal Control and Decision. Springer, London (in preparation) · Zbl 1154.49301 |
| [21] | Dontchev, A.L., Zolezzi, T.: Well-Posed Optimization Problems. Lecture Notes in Mathematics, vol. 1543. Springer, Berlin (1993) · Zbl 0797.49001 |
| [22] | Pinkus, A.: n-Widths in Approximation Theory. Springer, New York (1986) · Zbl 0593.41023 |
| [23] | Haykin, S.: Neural Networks: A Comprehensive Foundation. MacMillan, New York (1994) · Zbl 0828.68103 |
| [24] | Alessandri, A., Sanguineti, M.: Optimization of approximating networks for optimal fault diagnosis. Optim. Methods Softw. 20, 235–260 (2005) · Zbl 1072.90036 · doi:10.1080/10556780512331318245 |
| [25] | Alessandri, A., Cervellera, C., Sanguineti, M.: Functional optimal estimation problems and their approximate solution. J. Optim. Theory Appl. 134, 445–466 (2007) · Zbl 1151.93029 · doi:10.1007/s10957-007-9229-6 |
| [26] | Alessandri, A., Cervellera, C., Sanguineti, M.: Design of asymptotic estimators: An approach based on neural networks and nonlinear programming. IEEE Trans. Neural Netw. 18, 86–96 (2007) · Zbl 1151.93029 · doi:10.1109/TNN.2006.883015 |
| [27] | Singer, I.: Best Approximation in Normed Linear Spaces by Elements of Linear Subspaces. Springer, Berlin (1970) · Zbl 0197.38601 |
| [28] | Alessandri, A., Cuneo, M., Pagnan, S., Sanguineti, M.: A recursive algorithm for nonlinear least-squares problems. Comput. Optim. Appl. 38, 195–216 (2007) · Zbl 1144.62080 · doi:10.1007/s10589-007-9047-7 |
| [29] | Alessandri, A., Sanguineti, M., Maggiore, M.: Optimization-based learning with bounded error for feedforward neural networks. IEEE Trans. Neural Netw. 13, 261–273 (2002) · doi:10.1109/72.991413 |
| [30] | Bertsekas, D.P.: A new class of incremental gradient methods for least squares problems. SIAM J. Optim. 7, 913–926 (1997) · Zbl 0887.49025 · doi:10.1137/S1052623495287022 |
| [31] | Chow, T.W.S., Cho, S.-Y.: Neural Networks and Computing: Learning Algorithms and Applications. World Scientific, Singapore (2007) |
| [32] | Pinkus, A.: Approximation theory of the MLP model in neural networks. Acta Numer. 8, 143–196 (1999) · Zbl 0959.68109 · doi:10.1017/S0962492900002919 |
| [33] | Sanguineti, M.: Universal approximation by ridge computational models: A survey. Open Appl. Math. J. 2, 31–58 (2008) · Zbl 1322.68157 · doi:10.2174/1874114200802010031 |
| [34] | Park, J., Sandberg, I.W.: Universal approximation using radial-basis-function networks. Neural Comput. 3(2), 246–257 (1991) · doi:10.1162/neco.1991.3.2.246 |
| [35] | Friedman, A.: Foundations of Modern Analysis. Dover, New York (1982) · Zbl 0557.46001 |
| [36] | Stein, E.M.: Harmonic Analysis: Real-Variable Methods, Orthogonality, and Oscillatory Integrals. Princeton University Press, Princeton (1993) · Zbl 0821.42001 |
| [37] | Courant, R.: Differential and Integral Calculus, vol. II. Wiley-Interscience, New York (1988) · Zbl 0635.26001 |
| [38] | Adams, R.A.: Sobolev Spaces. Academic Press, New York (1975) · Zbl 0314.46030 |
| [39] | Stein, E.M.: Singular Integrals and Differentiability Properties of Functions. Princeton University Press, Princeton (1970) · Zbl 0207.13501 |
| [40] | Lebedev, N.N.: Special Functions and Their Applications. Dover, New York (1972) · Zbl 0271.33001 |
| [41] | Zolezzi, T.: Condition numbers and Ritz type methods in unconstrained optimization. Control Cybern. 36, 811–822 (2007) · Zbl 1166.49023 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
