On estimation of surrogate models for multivariate computer experiments. (English) Zbl 1415.62017
Summary: Estimation of surrogate models for computer experiments leads to nonparametric regression estimation problems without noise in the dependent variable. In this paper, we propose an empirical maximal deviation minimization principle to construct estimates in this context and analyze the rate of convergence of corresponding quantile estimates. As an application, we consider estimation of computer experiments with moderately high dimension by neural networks and show that here we can circumvent the so-called curse of dimensionality by imposing rather general assumptions on the structure of the regression function. The estimates are illustrated by applying them to simulated data and to a simulation model in mechanical engineering.
MSC:
| 62G08 | Nonparametric regression and quantile regression |
| 62G20 | Asymptotic properties of nonparametric inference |
| 62P30 | Applications of statistics in engineering and industry; control charts |
Keywords:
computer experiments; curse of dimensionality; neural networks; nonparametric regression without noise in the dependent variable; quantile estimates; rate of convergence; surrogate modelsSoftware:
ElemStatLearnReferences:
| [1] | Anthony, M., Bartlett, P. L. (1999). Neural networks and learning: Theoretical foundations. Cambridge: Cambridge University Press. · Zbl 0968.68126 |
| [2] | Barron, AR; Roussas, G. (ed.), Complexity regularization with application to artificial neural networks, 561-576 (1991), Dordrecht · Zbl 0739.62001 · doi:10.1007/978-94-011-3222-0_42 |
| [3] | Barron, A. R. (1993). Universal approximation bounds for superpositions of a sigmoidal function. IEEE Transactions on Information Theory, 39, 930-944. · Zbl 0818.68126 · doi:10.1109/18.256500 |
| [4] | Beirlant, J., Györfi, L. (1998). On the asymptotic \[{L}_2\] L2-error in partitioning regression estimation. Journal of Statistical Planning and Inference, 71, 93-107. · Zbl 0961.62030 |
| [5] | Bichon, B. J., Eldred, M. S., Swiler, L. P., Mahadevan, S., McFarland, J. M. (2008). Efficient global reliability analysis for nonlinear implicit performance functions. AIAA Journal, 46, 2459-2468. |
| [6] | Bourinet, J.-M., Deheeger, F., Lemaire, M. (2011). Assessing small failure probabilities by combined subset simulation and support vector machines. Structural Safety, 33, 343-353. |
| [7] | Bucher, C., Bourgund, U. (1990). A fast and efficient response surface approach for structural reliability problems. Structural Safety, 7, 57-66. |
| [8] | Das, P.-K., Zheng, Y. (2000). Cumulative formation of response surface and its use in reliability analysis. Probabilistic Engineering Mechanics, 15, 309-315. |
| [9] | Deheeger, F., Lemaire, M. (2010). Support vector machines for efficient subset simulations: \[^22\] SMART method. In: Proceedings of the 10th international conference on applications of statistics and probability in civil engineering (ICASP10), Tokyo, Japan. |
| [10] | Devroye, L. (1982). Necessary and sufficient conditions for the almost everywhere convergence of nearest neighbor regression function estimates. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete, 61, 467-481. · Zbl 0483.62029 · doi:10.1007/BF00531618 |
| [11] | Devroye, L., Krzyżak, A. (1989). An equivalence theorem for \[{L}_1\] L1 convergence of the kernel regression estimate. Journal of Statistical Planning and Inference, 23, 71-82. · Zbl 0686.62027 |
| [12] | Devroye, L., Wagner, T. J. (1980). Distribution-free consistency results in nonparametric discrimination and regression function estimation. Annals of Statistics, 8, 231-239. · Zbl 0431.62025 |
| [13] | Devroye, L., Györfi, L., Krzyżak, A., Lugosi, G. (1994). On the strong universal consistency of nearest neighbor regression function estimates. Annals of Statistics, 22, 1371-1385. · Zbl 0817.62038 |
| [14] | Devroye, L., Györfi, L., Lugosi, G. (1996). A probabilistic theory of pattern recognition. New York: Springer. · Zbl 0853.68150 |
| [15] | Enss, G., Kohler, M., Krzyżak, A., Platz, R. (2016). Nonparametric quantile estimation based on surrogate models. IEEE Transactions on Information Theory, 62, 5727-5739. · Zbl 1359.94377 |
| [16] | Friedman, J. H., Stuetzle, W. (1981). Projection pursuit regression. Journal of the American Statistical Association, 76, 817-823. |
| [17] | Greblicki, W., Pawlak, M. (1985). Fourier and Hermite series estimates of regression functions. Annals of the Institute of Statistical Mathematics, 37, 443-454. · Zbl 0623.62029 |
| [18] | Györfi, L. (1981). Recent results on nonparametric regression estimate and multiple classification. Problems of Control and Information Theory, 10, 43-52. · Zbl 0473.62032 |
| [19] | Györfi, L., Kohler, M., Krzyżak, A., Walk, H. (2002). A distribution-free theory of nonparametric regression. Springer series in statistics. New York: Springer. · Zbl 1021.62024 |
| [20] | Hansmann, M., Kohler, M. (2017). Estimation of quantiles from data with additional measurement errors. Statistica Sinica, 27, 1661-1673. · Zbl 1392.62093 |
| [21] | Hastie, T., Tibshirani, R., Friedman, J. (2011). The elements of statistical learning: Data mining, inference, and prediction (2nd ed.). New York: Springer. · Zbl 1273.62005 |
| [22] | Haykin, S. O. (2008). Neural networks and learning machines (3rd ed.). New York: Prentice-Hall. |
| [23] | Hertz, J., Krogh, A., Palmer, R. G. (1991). Introduction to the theory of neural computation. Redwood City, CA: Addison-Wesley. |
| [24] | Hurtado, J. (2004). Structural reliability—Statistical learning perspectives. Vol. 17 of lecture notes in applied and computational mechanics. Berlin: Springer. · Zbl 1086.62116 |
| [25] | Kaymaz, I. (2005). Application of Kriging method to structural reliability problems. Structural Safety, 27, 133-151. · doi:10.1016/j.strusafe.2004.09.001 |
| [26] | Kim, S.-H., Na, S.-W. (1997). Response surface method using vector projected sampling points. Structural Safety, 19, 3-19. |
| [27] | Kohler, M. (2000). Inequalities for uniform deviations of averages from expectations with applications to nonparametric regression. Journal of Statistical Planning and Inference, 89, 1-23. · Zbl 0982.62035 · doi:10.1016/S0378-3758(99)00215-3 |
| [28] | Kohler, M. (2014). Optimal global rates of convergence for noiseless regression estimation problems with adaptively chosen design. Journal of Multivariate Analysis, 132, 197-208. · Zbl 1360.62177 · doi:10.1016/j.jmva.2014.08.008 |
| [29] | Kohler, M., Krzyżak, A. (2001). Nonparametric regression estimation using penalized least squares. IEEE Transactions on Information Theory, 47, 3054-3058. · Zbl 1008.62580 |
| [30] | Kohler, M., & Krzyżak, A. (2005). Adaptive regression estimation with multilayer feedforward neural networks. Journal of Nonparametric Statistics, 17, 891-913. · Zbl 1121.62043 · doi:10.1080/10485250500309608 |
| [31] | Kohler, M., Krzyżak, A. (2013). Optimal global rates of convergence for interpolation problems with random design. Statistics and Probability Letters, 83, 1871-1879. · Zbl 1281.62121 |
| [32] | Kohler, M., Krzyżak, A. (2017). Nonparametric regression based on hierarchical interaction models. IEEE Transaction on Information Theory, 63, 1620-1630. · Zbl 1366.62082 |
| [33] | Lazzaro, D., Montefusco, L. (2002). Radial basis functions for the multivariate interpolation of large scattered data sets. Journal of Computational and Applied Mathematics, 140, 521-536. · Zbl 1025.65015 |
| [34] | Lugosi, G., Zeger, K. (1995). Nonparametric estimation via empirical risk minimization. IEEE Transactions on Information Theory, 41, 677-687. · Zbl 0818.62041 |
| [35] | Massart, P. (1990). The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality. Annals of Probability, 18, 1269-1283. · Zbl 0713.62021 · doi:10.1214/aop/1176990746 |
| [36] | McCaffrey, D. F., Gallant, A. R. (1994). Convergence rates for single hidden layer feedforward networks. Neural Networks, 7, 147-158. |
| [37] | Mhaskar, H. N. (1993). Approximation properties of multilayer feedforward artificial neural network. Advances in Computational Mathematics, 1, 61-80. · Zbl 0824.41011 · doi:10.1007/BF02070821 |
| [38] | Mielniczuk, J., Tyrcha, J. (1993). Consistency of multilayer perceptron regression estimators. Neural Networks, 6, 1019-1022. |
| [39] | Nadaraya, E. A. (1964). On estimating regression. Theory of Probability and Its Applications, 9, 141-142. · Zbl 0136.40902 · doi:10.1137/1109020 |
| [40] | Nadaraya, E. A. (1970). Remarks on nonparametric estimates for density functions and regression curves. Theory of Probability and Its Applications, 15, 134-137. · Zbl 0228.62031 · doi:10.1137/1115015 |
| [41] | Papadrakakis, M., Lagaros, N. (2002). Reliability-based structural optimization using neural networks and Monte Carlo simulation. Computer Methods in Applied Mechanics and Engineering, 191, 3491-3507. · Zbl 1101.74377 |
| [42] | Rafajłowicz, E. (1987). Nonparametric orthogonal series estimators of regression: A class attaining the optimal convergence rate in L2. Statistics and Probability Letters, 5, 219-224. · Zbl 0605.62030 · doi:10.1016/0167-7152(87)90044-7 |
| [43] | Ripley, B. D. (2008). Pattern recognition and neural networks. Cambridge: Cambridge University Press. · Zbl 1163.62047 |
| [44] | Stone, C. J. (1977). Consistent nonparametric regression. Annals of Statististics, 5, 595-645. · Zbl 0366.62051 · doi:10.1214/aos/1176343886 |
| [45] | Stone, C. J. (1982). Optimal global rates of convergence for nonparametric regression. Annals of Statistics, 10, 1040-1053. · Zbl 0511.62048 · doi:10.1214/aos/1176345969 |
| [46] | Stone, C. J. (1985). Additive regression and other nonparametric models. Annals of Statistics, 13, 689-705. · Zbl 0605.62065 · doi:10.1214/aos/1176349548 |
| [47] | Stone, C. J. (1994). The use of polynomial splines and their tensor products in multivariate function estimation. Annals of Statistics, 22, 118-184. · Zbl 0827.62038 · doi:10.1214/aos/1176325361 |
| [48] | Wahba, G. (1990). Spline models for observational data. Philadelphia, PA: SIAM. · Zbl 0813.62001 · doi:10.1137/1.9781611970128 |
| [49] | Watson, G. S. (1964). Smooth regression analysis. Sankhya Series A, 26, 359-372. · Zbl 0137.13002 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
