Asymptotic expansion for neural network operators of the Kantorovich type and high order of approximation. (English) Zbl 1469.41006
The rate of pointwise approximation for the neural network operators of the Kantorovich type has been studied by the authors. To improve the rate of convergence, they have considered finite linear combinations of the above neural network type operators. They have also discussed examples of sigmoidal activation functions.
Reviewer: Neha Malik (New Delhi)
MSC:
| 41A30 | Approximation by other special function classes |
| 41A05 | Interpolation in approximation theory |
| 41A25 | Rate of convergence, degree of approximation |
| 47A58 | Linear operator approximation theory |
Keywords:
sigmoidal functions; neural networks operators; asymptotic formula; Voronovskaja formula; high-order convergence; truncated algebraic momentReferences:
| [1] | Adell, JA; Cardenas-Morales, D., Quantitative generalized Voronovskaja’s formulae for Bernstein polynomials, J. Approx. Theory, 231, 41-52 (2018) · Zbl 1396.41013 · doi:10.1016/j.jat.2018.04.007 |
| [2] | Agarap, A.F.: Deep Learning using Rectified Linear Units (ReLU). arXiv:1803.08375 (2018) |
| [3] | Agostinelli, F., Hoffman, M., Sadowski, P., Baldi, P.: Learning Activation Functions to Improve Deep Neural Networks. arXiv:1412.6830v3 (2015) |
| [4] | Aral, A.; Acar, T.; Rasa, I., The new forms of Voronovskaya’s theorem in weighted spaces, Positivity, 20, 1, 25-40 (2016) · Zbl 1334.41015 · doi:10.1007/s11117-015-0338-4 |
| [5] | Asdrubali, F.; Baldinelli, G.; Bianchi, F.; Costarelli, D.; Rotili, A.; Seracini, M.; Vinti, G., Detection of thermal bridges from thermographic images by means of image processing approximation algorithms, Appl. Math. Comput., 317, 160-171 (2018) · Zbl 1426.94008 |
| [6] | Bardaro, C.; Mantellini, I., Voronovskaja formulae for Kantorovich generalized sampling series, Int. J. Pure Appl. Math., 62, 3, 247-262 (2010) · Zbl 1202.41011 |
| [7] | Bardaro, C.; Mantellini, I., Approximation properties for linear combinations of moment type operators, Comput. Math. Appl., 62, 5, 2304-2313 (2011) · Zbl 1231.41021 · doi:10.1016/j.camwa.2011.07.017 |
| [8] | Bardaro, C.; Mantellini, I., Asymptotic formulae for linear combinations of generalized sampling operators, Z. Anal. Ihre Anwend., 32, 3, 279-298 (2013) · Zbl 1279.41021 · doi:10.4171/ZAA/1485 |
| [9] | Barron, A.R., Klusowski, J.M.: Uniform approximation by neural networks activated by first and second order ridge splines. arXiv preprint arXiv:1607.07819 (2016) |
| [10] | Boccuto, A.; Bukhvalov, AV; Sambucini, AR, Some inequalities in classical spaces with mixed norms, Positivity, 6, 393-411 (2002) · Zbl 1043.46025 · doi:10.1023/A:1021353215312 |
| [11] | Butzer, PL; Nessel, RJ, Fourier Analysis and Approximation I (1971), New York: Academic Press, New York · Zbl 0217.42603 · doi:10.1007/978-3-0348-7448-9 |
| [12] | Cao, F.; Chen, Z., The approximation operators with sigmoidal functions, Comput. Math. Appl., 58, 4, 758-765 (2009) · Zbl 1189.41014 · doi:10.1016/j.camwa.2009.05.001 |
| [13] | Cao, F.; Chen, Z., Scattered data approximation by neural networks operators, Neurocomputing, 190, 237-242 (2016) · doi:10.1016/j.neucom.2016.01.013 |
| [14] | Cao, F.; Liu, B.; Park, DS, Image classification based on effective extreme learning machine, Neurocomputing, 102, 90-97 (2013) · doi:10.1016/j.neucom.2012.02.042 |
| [15] | Cardaliaguet, P.; Euvrard, G., Approximation of a function and its derivative with a neural network, Neural Netw., 5, 2, 207-220 (1992) · doi:10.1016/S0893-6080(05)80020-6 |
| [16] | Chui, C. K., Mhaskar, H. N.: Deep nets for local manifold learning. arXiv preprint arXiv:1607.07110 (2016) |
| [17] | Coroianu, L.; Gal, SG, Approximation by truncated max-product operators of Kantorovich-type based on generalized \((\varphi ,\psi )\)-kernels, Math. Methods Appl. Sci., 41, 17, 7971-7984 (2018) · Zbl 1405.41012 · doi:10.1002/mma.5262 |
| [18] | Costarelli, D.: Approximate solutions of Volterra integral equations by an interpolation method based on ramp functions. Comput. Appl. Math. 38(4) (article 159) (2019) · Zbl 1438.65323 |
| [19] | Costarelli, D.; Minotti, AM; Vinti, G., Approximation of discontinuous signals by sampling Kantorovich series, J. Math. Anal. Appl., 450, 2, 1083-1103 (2017) · Zbl 1373.41018 · doi:10.1016/j.jmaa.2017.01.066 |
| [20] | Costarelli, D.; Sambucini, AR; Vinti, G., Convergence in Orlicz spaces by means of the multivariate max-product neural network operators of the Kantorovich type and applications, Neural Comput. Appl., 31, 5069-5078 (2019) · doi:10.1007/s00521-018-03998-6 |
| [21] | Costarelli, D.; Spigler, R., Convergence of a family of neural network operators of the Kantorovich type, J. Approx. Theory, 185, 80-90 (2014) · Zbl 1297.41003 · doi:10.1016/j.jat.2014.06.004 |
| [22] | Costarelli, D.; Spigler, R., Approximation by series of sigmoidal functions with applications to neural networks, Ann. Mat. Pura Appl., 194, 1, 289-306 (2015) · Zbl 1321.41021 · doi:10.1007/s10231-013-0378-y |
| [23] | Costarelli, D.; Vinti, G., Pointwise and uniform approximation by multivariate neural network operators of the max-product type, Neural Netw., 81, 81-90 (2016) · Zbl 1439.41009 · doi:10.1016/j.neunet.2016.06.002 |
| [24] | Costarelli, D.; Vinti, G., Convergence for a family of neural network operators in Orlicz spaces, Math. Nachr., 290, 2-3, 226-235 (2017) · Zbl 1373.47010 · doi:10.1002/mana.201600006 |
| [25] | Costarelli, D.; Vinti, G., Convergence results for a family of Kantorovich max-product neural network operators in a multivariate setting, Math. Slov., 67, 6, 1469-1480 (2017) · Zbl 1505.41005 |
| [26] | Costarelli, D.; Vinti, G., Saturation classes for max-product neural network operators activated by sigmoidal functions, Results Math., 72, 3, 1555-1569 (2017) · Zbl 1376.41014 · doi:10.1007/s00025-017-0692-6 |
| [27] | Costarelli, D.; Vinti, G., Inverse results of approximation and the saturation order for the sampling Kantorovich series, J. Approx. Theory, 242, 64-82 (2019) · Zbl 1416.41019 · doi:10.1016/j.jat.2019.03.001 |
| [28] | Costarelli, D., Vinti, G.: Voronovskaja formulas for high order convergence neural network operator with sigmoidal functions. Mediterr. J. Math. 17 (article numb. 77), doi:10.1007/s00009-020-01513-7 (2020) · Zbl 1447.41007 |
| [29] | Cucker, F.; Zhou, DX, Learning Theory an Approximation Theory Viewpoint (2007), Cambridge: Cambridge University Press, Cambridge · Zbl 1274.41001 · doi:10.1017/CBO9780511618796 |
| [30] | Cybenko, G., Approximation by superpositions of a sigmoidal function, Math. Control Signals Syst., 2, 303-314 (1989) · Zbl 0679.94019 · doi:10.1007/BF02551274 |
| [31] | Eckle, K.; Schmidt-Hieber, J., A comparison of deep networks with ReLU activation function and linear spline-type methods, Neural Netw., 110, 232-242 (2019) · doi:10.1016/j.neunet.2018.11.005 |
| [32] | Fard, S.P., Zainuddin, Z.: The universal approximation capabilities of cylindrical approximate identity neural networks. Arab. J. Sci. Eng. 1-8 (2016) · Zbl 1390.92015 |
| [33] | Fard, S.P., Zainuddin, Z.: Theoretical analyses of the universal approximation capability of a class of higher order neural networks based on approximate identity. In: Nature-Inspired Computing: Concepts, Methodologies, Tools, and Applications (in print). doi:10.4018/978-1-5225-0788-8.ch055 (2016) · Zbl 1390.92015 |
| [34] | Finta, Z., On generalized Voronovskaja theorem for Bernstein polynomials, Carpathian J. Math., 28, 2, 231-238 (2012) · Zbl 1299.41041 |
| [35] | Gavrea, I.; Ivan, M., The Bernstein Voronovskaja-type theorem for positive linear approximation operators, J. Approx. Theory, 192, 291-296 (2015) · Zbl 1310.41003 · doi:10.1016/j.jat.2014.12.008 |
| [36] | Goebbels, S.: On sharpness of error bounds for single hidden layer feedforward neural networks. arXiv:1811.05199 (2018) · Zbl 1443.62314 |
| [37] | Gripenberg, G., Approximation by neural network with a bounded number of nodes at each level, J. Approx. Theory, 122, 2, 260-266 (2003) · Zbl 1019.41018 · doi:10.1016/S0021-9045(03)00078-9 |
| [38] | Guliyev, NJ; Ismailov, VE, On the approximation by single hidden layer feedforward neural networks with fixed weights, Neural Netw., 98, 296-304 (2018) · Zbl 1437.68062 · doi:10.1016/j.neunet.2017.12.007 |
| [39] | Guliyev, NJ; Ismailov, VE, Approximation capability of two hidden layer feedforward neural networks with fixed weights, Neurocomputing, 316, 262-269 (2018) · doi:10.1016/j.neucom.2018.07.075 |
| [40] | Iliev, A.; Kyurkchiev, N., On the Hausdor distance between the Heaviside function and some transmuted activation functions, Math. Model. Appl., 2, 1, 1-5 (2016) |
| [41] | Iliev, A.; Kyurkchiev, N.; Markov, S., On the approximation of the cut and step functions by logistic and Gompertz functions, BIOMATH, 4, 2, 1510101 (2015) · Zbl 1369.41025 · doi:10.11145/j.biomath.2015.10.101 |
| [42] | Ismailov, VE, On the approximation by neural networks with bounded number of neurons in hidden layers, J. Math. Anal. Appl., 417, 2, 963-969 (2014) · Zbl 1303.41010 · doi:10.1016/j.jmaa.2014.03.092 |
| [43] | Kainen, PC; Kurkovà, V., An integral upper bound for neural network approximation, Neural Comput., 21, 2970-2989 (2009) · Zbl 1186.68370 · doi:10.1162/neco.2009.04-08-745 |
| [44] | Kurkovà, V., Lower bounds on complexity of shallow perceptron networks, Eng. Appl. Neural Netw. Commun. Comput. Inform. Sci., 629, 283-294 (2016) |
| [45] | Lin, S., Zeng, J., Zhang, X.: Constructive neural network learning. arXiv preprint arXiv:1605.00079 (2016) |
| [46] | Maiorov, V.; Meir, R., On the near optimality of the stochastic approximation of smooth functions by neural networks, Adv. Comput. Math., 13, 1, 79-103 (2000) · Zbl 0939.41013 · doi:10.1023/A:1018993908478 |
| [47] | Makovoz, Y., Uniform approximation by neural networks, J. Approx. Theory, 95, 2, 215-228 (1998) · Zbl 0932.41016 · doi:10.1006/jath.1997.3217 |
| [48] | Mhaskar, H., Poggio, T.: Deep vs. shallow networks: an approximation theory perspective. Anal. Appl. 14 (6), 829-848 (2016) · Zbl 1355.68233 |
| [49] | Moritani, Y.; Ogihara, N., A hypothetical neural network model for generation of human precision grip, Neural Netw., 110, 213-224 (2019) · doi:10.1016/j.neunet.2018.12.001 |
| [50] | Nasaireh, F.; Rasa, I., Another look at Voronovskaja type formulas, J. Math. Inequal., 12, 1, 95-105 (2018) · Zbl 1391.41016 · doi:10.7153/jmi-2018-12-07 |
| [51] | Petersen, P.; Voigtlaender, F., Optimal approximation of piecewise smooth functions using deep ReLU neural networks, Neural Netw., 108, 296-330 (2018) · Zbl 1434.68516 · doi:10.1016/j.neunet.2018.08.019 |
| [52] | Schmidhuber, J., Deep learning in neural networks: an overview, Neural Netw., 61, 85-117 (2015) · doi:10.1016/j.neunet.2014.09.003 |
| [53] | Smale, S.; Zhou, DX, Learning theory estimates via integral operators and their approximations, Construct. Approx., 26, 2, 153-172 (2007) · Zbl 1127.68088 · doi:10.1007/s00365-006-0659-y |
| [54] | Ulusoy, G., Acar, T.: q-Voronovskaya type theorems for q-Baskakov operators. Math. Methods Appl. Sci. (2015). doi:10.1002/mma.3784 · Zbl 1347.41030 |
| [55] | Vinti, G.; Zampogni, L., A unifying approach to convergence of linear sampling type operators in Orlicz spaces, Adv. Differ. Equ., 16, 5-6, 573-600 (2011) · Zbl 1223.41014 |
| [56] | Zhang, Y.; Wu, J.; Cai, Z.; Du, B.; Yu, PS, An unsupervised parameter learning model for RVFL neural network, Neural Netw., 112, 85-97 (2019) · Zbl 1435.68289 · doi:10.1016/j.neunet.2019.01.007 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
