Computationally efficient algorithm for Gaussian process regression in case of structured samples. (English. Russian original) Zbl 1403.62141
Comput. Math. Math. Phys. 56, No. 4, 499-513 (2016); translation from Zh. Vychisl. Mat. Mat. Fiz. 56, No. 4, 507-522 (2016).
Summary: Surrogate modeling is widely used in many engineering problems. Data sets often have Cartesian product structure (for instance factorial design of experiments with missing points). In such case the size of the data set can be very large. Therefore, one of the most popular algorithms for approximation – Gaussian process regression – can be hardly applied due to its computational complexity. In this paper a computationally efficient approach for constructing Gaussian process regression in case of data sets with Cartesian product structure is presented. Efficiency is achieved by using a special structure of the data set and operations with tensors. Proposed algorithm has low computational as well as memory complexity compared to existing algorithms. In this work we also introduce a regularization procedure allowing to take into account anisotropy of the data set and avoid degeneracy of regression model.
MSC:
| 62K15 | Factorial statistical designs |
| 60G15 | Gaussian processes |
| 62J02 | General nonlinear regression |
Keywords:
Gaussian processes; regularization; factorial design of experiments; incomplete factorial design of experiments; operations with tensorsReferences:
| [1] | A. Forrester, A. Sobester, and A. Keane, Engineering Design via Surrogate Modeling: A Practical Guide (Wiley, New York, 2008). · doi:10.1002/9780470770801 |
| [2] | C. E. Rasmussen and C. Williams, Gaussian Processes for Machine Learning (MIT Press, Cambridge, Mass., 2006). · Zbl 1177.68165 |
| [3] | Rasmussen, C. E.; Ghahramani, Z., Infinite mixtures of Gaussian process experts, 881-888 (2001), Cambridge, Mass. |
| [4] | J. Quiñonero-Candela and C. E. Rasmussen, “A unifying view of sparse approximate Gaussian process regression,” J. Mach. Learn. Res. 6, 1939-1959 2005. · Zbl 1222.68282 |
| [5] | D. C. Montgomery, Design and Analysis of Experiments (Wiley, New York, 2006). |
| [6] | J. S. Charles, M. Hansen, C. Kooperberg, and Y. K. Truong, “Polynomial splines and their tensor products in extended linear modeling,” Ann. Stat. 25, 1371-1470 1997. · Zbl 0924.62036 · doi:10.1214/aos/1031594728 |
| [7] | C. R. Dietrich and G. N. Newsam, “Fast and exact simulation of stationary Gaussian processes through circulant embedding of the covariance matrix,” SIAM J. Sci. Comput. 18, 1088-1107 1997. · Zbl 0890.65149 · doi:10.1137/S1064827592240555 |
| [8] | D. L. Zimmerman, “Computationally efficient restricted maximum likelihood estimation of generalized covariance functions,” Math. Geol. 21 (7), 655-672 1989. · Zbl 0970.65500 · doi:10.1007/BF00893314 |
| [9] | G. Chan and A. T. A. Wood, “Algorithm AS 312: An algorithm for simulating stationary Gaussian random fields,” J. R. Stat. Soc. Ser. C (Appl. Stat.) 46 (1), 171-181 1997. · Zbl 0913.65142 · doi:10.1111/1467-9876.00057 |
| [10] | T. C. S. Rendall and C. B. Allen, “Multi-dimensional aircraft surface pressure interpolation using radial basis functions,” Proc. IMechE Part G. Aerospace Eng. 222, 483-495 2008. · doi:10.1243/09544100JAERO263 |
| [11] | A. N. Shiryaev, Probability-1: Elementary Probability Theory, Mathematical Foundations, and Limit Theorems (MTsNMO, Moscow, 2011) [in Russian]. |
| [12] | G. K. Tamara and W. B. Brett, “Tensor decompositions and applications,” SIAM Rev. 51 (3), 455-500 2009. · Zbl 1173.65029 · doi:10.1137/07070111X |
| [13] | R. A. Horn and C. R. Johnson, Topics in Matrix Analysis (Cambridge Univ. Press, Cambridge, 1994). · Zbl 0801.15001 |
| [14] | C. E. Rasmussen and H. Nickisch, “Gaussian processes for machine learning (GPML) toolbox,” J. Mach. Learn. Res. 11, 3011-3015 2010. · Zbl 1242.68242 |
| [15] | J. Nocedal and S. J. Wright, Numerical Optimization, 2nd ed. (Springer, New York, 2006). · Zbl 1104.65059 |
| [16] | Evolutionary computation pages: the function testbed. http://wwwtikeeethzch/sop/download/supplementary/testproblems/ |
| [17] | System optimization: test problems. http://wwwtikeeethzch/sop/download/supplementary/testproblems/ |
| [18] | Snelson, E.; Ghahramani, Z., Sparse Gaussian processes using pseudo-inputs, 1257-1264 (2005), Cambridge, Mass. |
| [19] | J. H. Friedman, “Multivariate adaptive regression splines,” Ann. Stat. 19 (1), 1-67 1991. · Zbl 0765.62064 · doi:10.1214/aos/1176347963 |
| [20] | E. D. Dolan and J. J. Moré, “Benchmarking optimization software with performance profiles,” Math. Program. 91 (2), 201-213 2002. · Zbl 1049.90004 · doi:10.1007/s101070100263 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
