Posterior consistency for Gaussian process approximations of Bayesian posterior distributions. (English) Zbl 1429.60040
Summary: We study the use of Gaussian process emulators to approximate the parameter-to-observation map or the negative log-likelihood in Bayesian inverse problems. We prove error bounds on the Hellinger distance between the true posterior distribution and various approximations based on the Gaussian process emulator. Our analysis includes approximations based on the mean of the predictive process, as well as approximations based on the full Gaussian process emulator. Our results show that the Hellinger distance between the true posterior and its approximations can be bounded by moments of the error in the emulator. Numerical results confirm our theoretical findings.
MSC:
| 60G15 | Gaussian processes |
| 62G08 | Nonparametric regression and quantile regression |
| 65D05 | Numerical interpolation |
| 65D30 | Numerical integration |
| 65J22 | Numerical solution to inverse problems in abstract spaces |
Keywords:
inverse problem; Bayesian approach; surrogate model; Gaussian process regression; posterior consistencyReferences:
| [1] | Adler, Robert J., The Geometry of Random Fields, xi+280 pp. (1981), John Wiley & Sons, Ltd., Chichester · Zbl 1182.60017 |
| [2] | Andrieu, Christophe; Roberts, Gareth O., The pseudo-marginal approach for efficient Monte Carlo computations, Ann. Statist., 37, 2, 697-725 (2009) · Zbl 1185.60083 · doi:10.1214/07-AOS574 |
| [3] | Aronszajn, N., Theory of reproducing kernels, Trans. Amer. Math. Soc., 68, 337-404 (1950) · Zbl 0037.20701 |
| [4] | Arridge, S. R.; Kaipio, J. P.; Kolehmainen, V.; Schweiger, M.; Somersalo, E.; Tarvainen, T.; Vauhkonen, M., Approximation errors and model reduction with an application in optical diffusion tomography, Inverse Problems, 22, 1, 175-195 (2006) · Zbl 1138.65042 · doi:10.1088/0266-5611/22/1/010 |
| [5] | Babu{\v{s}}ka, Ivo; Nobile, Fabio; Tempone, Ra{\'u}l, A stochastic collocation method for elliptic partial differential equations with random input data, SIAM Rev., 52, 2, 317-355 (2010) · Zbl 1226.65004 · doi:10.1137/100786356 |
| [6] | bzkl13 I. Bilionis, N. Zabaras, B. A. Konomi, and G. Lin, Multi-output separable Gaussian process: Towards an efficient, fully Bayesian paradigm for uncertainty quantification, Journal of Computational Physics 241 (2013), 212-239. · Zbl 1349.76760 |
| [7] | Bliznyuk, Nikolay; Ruppert, David; Shoemaker, Christine; Regis, Rommel; Wild, Stefan; Mugunthan, Pradeep, Bayesian calibration and uncertainty analysis for computationally expensive models using optimization and radial basis function approximation, J. Comput. Graph. Statist., 17, 2, 270-294 (2008) · doi:10.1198/106186008X320681 |
| [8] | Bogachev, Vladimir I., Gaussian Measures, Mathematical Surveys and Monographs 62, xii+433 pp. (1998), American Mathematical Society, Providence, RI · Zbl 0913.60035 · doi:10.1090/surv/062 |
| [9] | Bui-Thanh, T.; Willcox, K.; Ghattas, O., Model reduction for large-scale systems with high-dimensional parametric input space, SIAM J. Sci. Comput., 30, 6, 3270-3288 (2008) · Zbl 1196.37127 · doi:10.1137/070694855 |
| [10] | Cohen, Albert; Devore, Ronald; Schwab, Christoph, Analytic regularity and polynomial approximation of parametric and stochastic elliptic PDE’s, Anal. Appl. (Singap.), 9, 1, 11-47 (2011) · Zbl 1219.35379 · doi:10.1142/S0219530511001728 |
| [11] | cmps14 P. R. Conrad, Y. M. Marzouk, N. S. Pillai, and A. Smith, Asymptotically exact MCMC algorithms via local approximations of computationally intensive models, J. Amer. Statist. Assoc. 111 (2016), 1591-1607. |
| [12] | Constantine, Paul G., Active Subspaces, SIAM Spotlights 2, ix+100 pp. (2015), Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA · Zbl 1431.65001 |
| [13] | Constantine, Paul G.; Dow, Eric; Wang, Qiqi, Active subspace methods in theory and practice: applications to kriging surfaces, SIAM J. Sci. Comput., 36, 4, A1500-A1524 (2014) · Zbl 1311.65008 · doi:10.1137/130916138 |
| [14] | Cotter, S. L.; Roberts, G. O.; Stuart, A. M.; White, D., MCMC methods for functions: modifying old algorithms to make them faster, Statist. Sci., 28, 3, 424-446 (2013) · Zbl 1331.62132 · doi:10.1214/13-STS421 |
| [15] | ds15 M. Dashti and A.M.Stuart, The Bayesian Approach to Inverse Problems, Handbook of Uncertainty Quantification (R. Ghanem, D. Higdon, and H. Owhadi, eds.), Springer. |
| [16] | Girolami, Mark; Calderhead, Ben, Riemann manifold Langevin and Hamiltonian Monte Carlo methods, J. R. Stat. Soc. Ser. B Stat. Methodol., 73, 2, 123-214 (2011) · Zbl 1411.62071 · doi:10.1111/j.1467-9868.2010.00765.x |
| [17] | Hansen, Markus; Schwab, Christoph, Sparse adaptive approximation of high dimensional parametric initial value problems, Vietnam J. Math., 41, 2, 181-215 (2013) · Zbl 1272.34012 · doi:10.1007/s10013-013-0011-9 |
| [18] | Hastings, W. K., Monte Carlo sampling methods using Markov chains and their applications, Biometrika, 57, 1, 97-109 (1970) · Zbl 0219.65008 · doi:10.1093/biomet/57.1.97 |
| [19] | Higdon, Dave; Kennedy, Marc; Cavendish, James C.; Cafeo, John A.; Ryne, Robert D., Combining field data and computer simulations for calibration and prediction, SIAM J. Sci. Comput., 26, 2, 448-466 (2004) · Zbl 1072.62018 · doi:10.1137/S1064827503426693 |
| [20] | Kaipio, Jari; Somersalo, Erkki, Statistical and Computational Inverse Problems, Applied Mathematical Sciences 160, xvi+339 pp. (2005), Springer-Verlag, New York · Zbl 1068.65022 |
| [21] | Kennedy, Marc C.; O’Hagan, Anthony, Bayesian calibration of computer models, J. R. Stat. Soc. Ser. B Stat. Methodol., 63, 3, 425-464 (2001) · Zbl 1007.62021 · doi:10.1111/1467-9868.00294 |
| [22] | Marzouk, Youssef; Xiu, Dongbin, A stochastic collocation approach to Bayesian inference in inverse problems, Commun. Comput. Phys., 6, 4, 826-847 (2009) · Zbl 1364.62064 · doi:10.4208/cicp.2009.v6.p826 |
| [23] | Marzouk, Youssef M.; Najm, Habib N.; Rahn, Larry A., Stochastic spectral methods for efficient Bayesian solution of inverse problems, J. Comput. Phys., 224, 2, 560-586 (2007) · Zbl 1120.65306 · doi:10.1016/j.jcp.2006.10.010 |
| [24] | Mat{\'e}rn, Bertil, Spatial variation, Lecture Notes in Statistics 36, 151 pp. (1986), Springer-Verlag, Berlin · Zbl 0608.62122 · doi:10.1007/978-1-4615-7892-5 |
| [25] | mercer09 J. Mercer, Functions of positive and negative type, and their connection with the theory of integral equations, Philosophical Transactions of the Royal Society of London, Series A 209 (1909), 415-446. · JFM 40.0408.02 |
| [26] | mrrtt53 N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller, Equation of state calculations by fast computing machines, J. Chemical Physics 21 (1953), 1087. · Zbl 1431.65006 |
| [27] | Narcowich, Francis J.; Ward, Joseph D.; Wendland, Holger, Sobolev bounds on functions with scattered zeros, with applications to radial basis function surface fitting, Math. Comp., 74, 250, 743-763 (2005) · Zbl 1063.41013 · doi:10.1090/S0025-5718-04-01708-9 |
| [28] | Narcowich, Francis J.; Ward, Joseph D.; Wendland, Holger, Sobolev error estimates and a Bernstein inequality for scattered data interpolation via radial basis functions, Constr. Approx., 24, 2, 175-186 (2006) · Zbl 1120.41022 · doi:10.1007/s00365-005-0624-7 |
| [29] | Niederreiter, Harald, Random Number Generation and Quasi-Monte Carlo Methods, CBMS-NSF Regional Conference Series in Applied Mathematics 63, vi+241 pp. (1992), Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA · Zbl 0761.65002 · doi:10.1137/1.9781611970081 |
| [30] | o2006bayesian A. O’Hagan, Bayesian analysis of computer code outputs: a tutorial, Reliability Engineering & System Safety 91 (2006), no. 10, 1290-1300. |
| [31] | Da Prato, Giuseppe; Zabczyk, Jerzy, Stochastic Equations in Infinite Dimensions, Encyclopedia of Mathematics and its Applications 44, xviii+454 pp. (1992), Cambridge University Press, Cambridge · Zbl 1140.60034 · doi:10.1017/CBO9780511666223 |
| [32] | Rasmussen, Carl Edward; Williams, Christopher K. I., Gaussian Processes for Machine Learning, Adaptive Computation and Machine Learning, xviii+248 pp. (2006), MIT Press, Cambridge, MA · Zbl 1177.68165 |
| [33] | Rebeschini, Patrick; van Handel, Ramon, Can local particle filters beat the curse of dimensionality?, Ann. Appl. Probab., 25, 5, 2809-2866 (2015) · Zbl 1325.60058 · doi:10.1214/14-AAP1061 |
| [34] | Robert, Christian P.; Casella, George, Monte Carlo statistical methods, Springer Texts in Statistics, xxii+507 pp. (1999), Springer-Verlag, New York · Zbl 0935.62005 · doi:10.1007/978-1-4757-3071-5 |
| [35] | Rudin, Walter, Principles of Mathematical Analysis, Second edition, ix+270 pp. (1964), McGraw-Hill Book Co., New York · Zbl 0148.02903 |
| [36] | Sacks, Jerome; Welch, William J.; Mitchell, Toby J.; Wynn, Henry P., Design and analysis of computer experiments, Statist. Sci., 4, 4, 409-435 (1989) · Zbl 0955.62619 |
| [37] | Scheuerer, M.; Schaback, R.; Schlather, M., Interpolation of spatial data-a stochastic or a deterministic problem?, European J. Appl. Math., 24, 4, 601-629 (2013) · Zbl 1426.62284 · doi:10.1017/S0956792513000016 |
| [38] | Schillings, Cl.; Schwab, Ch., Sparsity in Bayesian inversion of parametric operator equations, Inverse Problems, 30, 6, 065007, 30 pp. (2014) · Zbl 1291.65033 · doi:10.1088/0266-5611/30/6/065007 |
| [39] | sn16 M. Sinsbeck and W. Nowak, Sequential design of computer experiments for the solution of Bayesian inverse problems with process emulators, SIAM/ASA Journal on Uncertainty Quantification, to appear · Zbl 1387.62093 |
| [40] | Stein, Michael L., Interpolation of Spatial Data, Springer Series in Statistics, xviii+247 pp. (1999), Springer-Verlag, New York · Zbl 0924.62100 · doi:10.1007/978-1-4612-1494-6 |
| [41] | Stuart, A. M., Inverse problems: a Bayesian perspective, Acta Numer., 19, 451-559 (2010) · Zbl 1242.65142 · doi:10.1017/S0962492910000061 |
| [42] | Walter, Wolfgang, Ordinary differential equations, Graduate Texts in Mathematics 182, xii+380 pp. (1998), Springer-Verlag, New York · Zbl 0991.34001 · doi:10.1007/978-1-4612-0601-9 |
| [43] | Wendland, Holger, Scattered Data Approximation, Cambridge Monographs on Applied and Computational Mathematics 17, x+336 pp. (2005), Cambridge University Press, Cambridge · Zbl 1075.65021 |
| [44] | Wu, Zong Min; Schaback, Robert, Local error estimates for radial basis function interpolation of scattered data, IMA J. Numer. Anal., 13, 1, 13-27 (1993) · Zbl 0762.41006 · doi:10.1093/imanum/13.1.13 |
| [45] | Xiu, Dongbin; Karniadakis, George Em, Modeling uncertainty in flow simulations via generalized polynomial chaos, J. Comput. Phys., 187, 1, 137-167 (2003) · Zbl 1047.76111 · doi:10.1016/S0021-9991(03)00092-5 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
