Non-stationary multi-layered Gaussian priors for Bayesian inversion. (English) Zbl 1457.60097
Summary: In this article, we study Bayesian inverse problems with multi-layered Gaussian priors. The aim of the multi-layered hierarchical prior is to provide enough complexity structure to allow for both smoothing and edge-preserving properties at the same time. We first describe the conditionally Gaussian layers in terms of a system of stochastic partial differential equations. We then build the computational inference method using a finite-dimensional Galerkin method. We show that the proposed approximation has a convergence-in-probability property to the solution of the original multi-layered model. We then carry out Bayesian inference using the preconditioned Crank-Nicolson algorithm which is modified to work with multi-layered Gaussian fields. We show via numerical experiments in signal deconvolution and computerized x-ray tomography problems that the proposed method can offer both smoothing and edge preservation at the same time.
MSC:
| 60H15 | Stochastic partial differential equations (aspects of stochastic analysis) |
| 62M99 | Inference from stochastic processes |
Keywords:
stochastic partial differential equation; inverse problem; Bayesian inverse problem; multi-layer Gaussian field priorsReferences:
| [1] | Kaipio J and Somersalo E 2004 Statistical and Computational Inverse Problems (Berlin: Springer) |
| [2] | Rasmussen C E and Williams C K I 2005 Gaussian Processes for Machine Learning (Cambridge, MA: MIT Press Ltd) · doi:10.7551/mitpress/3206.001.0001 |
| [3] | Heaton M J et al 2018 A case study competition among methods for analyzing large spatial data J. Agric. Biol. Environ. Stat.24 398-425 · Zbl 1426.62345 · doi:10.1007/s13253-018-00348-w |
| [4] | Paciorek C J and Schervish M J 2004 Nonstationary covariance functions for Gaussian process regression Adv. Neur. Inf. Proc. Sys. 273-80 |
| [5] | Fuglstad G-A, Simpson D, Lindgren F and Rue H 2015 Does non-stationary spatial data always require non-stationary random fields? Spat. Stat.14 505-31 · doi:10.1016/j.spasta.2015.10.001 |
| [6] | Paciorek C J 2003 Non-Stationary Gaussian processes for regression and spatial modelling PhD Thesis |
| [7] | Snelson E, Ghahramani Z and Rasmussen C E 2004 Warped Gaussian processes in Adv. Neur. Inf. Proc. Sys. 337-44 |
| [8] | Sampson P D and Guttorp P 1992 Nonparametric estimation of nonstationary spatial covariance structure J. Am. Stat. Assoc.87 108-19 · doi:10.1080/01621459.1992.10475181 |
| [9] | Green P J 1995 Reversible jump Markov chain Monte Carlo computation and Bayesian model determination Biometrika82 711-32 · Zbl 0861.62023 · doi:10.1093/biomet/82.4.711 |
| [10] | Ray A and Myer D 2019 Bayesian geophysical inversion with trans-dimensional Gaussian process machine learning Geophys. J. Int.217 1706-26 · doi:10.1093/gji/ggz111 |
| [11] | Lindgren F, Rue H and Lindström J 2011 An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach J. Roy. Stat. Soc. B 73 423-98 · Zbl 1274.62360 · doi:10.1111/j.1467-9868.2011.00777.x |
| [12] | Roininen L, Girolami M, Lasanen S and Markkanen M 2019 Hyperpriors for Matérn fields with applications in Bayesian inversion Inverse Problems Imaging13 1-29 · Zbl 1454.60068 · doi:10.3934/ipi.2019001 |
| [13] | Monterrubio-Gómez K, Roininen L, Wade S, Damoulas T and Girolami M 2020 Posterior inference for sparse hierarchical non-stationary models Comput. Stat. Data Anal.148 106954 · Zbl 1510.62068 · doi:10.1016/j.csda.2020.106954 |
| [14] | Damianou A C and Lawrence N D 2013 Deep gaussian processes (Artificial Intelligence and Statistics) pp 207-15 |
| [15] | Duvenaud D, Rippel O, Adams R P and Ghahramani Z 2014 Avoiding pathologies in very deep networks Artificial Intelligence and Statistics pp 202-10 |
| [16] | Dunlop M M, Girolami M A, Stuart A M and Teckentrup A L 2018 How deep are deep Gaussian processes? J. Mach. Learn. Res.19 1-46 · Zbl 1469.60107 |
| [17] | Stuart A M 2010 Inverse problems: a Bayesian perspective Acta Numerica19 451-559 · Zbl 1242.65142 · doi:10.1017/s0962492910000061 |
| [18] | Dashti M and Stuart A M 2017 The Bayesian approach to inverse problems Handbook of Uncertainty Quantification (Berlin: Springer International Publishing) pp 311-428 · doi:10.1007/978-3-319-12385-1_7 |
| [19] | Cotter S L, Roberts G O, Stuart A M and White D 2013 MCMC methods for functions: modifying old algorithms to make them faster Stat. Sci.28 424-46 · Zbl 1331.62132 · doi:10.1214/13-sts421 |
| [20] | Law K J H 2014 Proposals which speed up function-space MCMC J. Comput. Appl. Math.262 127-38 · Zbl 1301.65004 · doi:10.1016/j.cam.2013.07.026 |
| [21] | Beskos A, Girolami M, Lan S, Farrell P E and Stuart A M 2017 Geometric MCMC for infinite-dimensional inverse problems J. Comput. Phys.335 327-51 · Zbl 1375.35627 · doi:10.1016/j.jcp.2016.12.041 |
| [22] | Rudolf D and Sprungk B 2018 On generalization of the preconditioned Crank-Nicolson Metropolis algorithm Found. Comput. Math.18 309-43 · Zbl 1391.60169 · doi:10.1007/s10208-016-9340-x |
| [23] | Hairer M, Stuart A M and Vollmer S J 2014 Spectral gaps for a Metropolis-Hastings algorithm in infinite dimensions Ann. Appl. Probab.24 2455-90 · Zbl 1307.65002 · doi:10.1214/13-aap982 |
| [24] | Chen V, Dunlop M M, Papaspiliopoulos O and Stuart A M 2018 Dimension-robust MCMC in Bayesian inverse problems (arXiv 1803.03344) |
| [25] | Papaspiliopoulos O, Roberts G O and Sköld M 2007 A general framework for the parametrization of hierarchical models Stat. Sci.22 59-73 · Zbl 1246.62195 · doi:10.1214/088342307000000014 |
| [26] | Hu Z, Yao Z and Li J 2017 On an adaptive preconditioned Crank-Nicolson MCMC algorithm for infinite dimensional bayesian inference J. Comput. Phys.332 492-503 · Zbl 1380.65011 · doi:10.1016/j.jcp.2016.11.024 |
| [27] | Bernardo J M and Smith A F M 2000 Bayesian Theory (New York: Wiley) · Zbl 0943.62009 |
| [28] | Emzir M, Lasanen S, Purisha Z and Särkkä S 2019 Hilbert-space reduced-rank methods for deep Gaussian processes Proc. of IEEE Int. Workshop on Machine Learning for Signal Processing (MLSP) · doi:10.1109/MLSP.2019.8918874 |
| [29] | Solin A and Särkkä S 2019 Hilbert space methods for reduced-rank Gaussian process regression Stat. Comput.30 419-46 · Zbl 1436.62316 · doi:10.1007/s11222-019-09886-w |
| [30] | Blei D M, Kucukelbir A and McAuliffe J D 2017 Variational inference: a review for statisticians J. Am. Stat. Assoc.112 859-77 · doi:10.1080/01621459.2017.1285773 |
| [31] | Beal M J 2003 Variational algorithms for approximate Bayesian inference PhD Thesis University of London |
| [32] | Wang Y and Blei D M 2018 Frequentist consistency of variational Bayes J. Am. Stat. Assoc.114 1147-61 · Zbl 1428.62119 · doi:10.1080/01621459.2018.1473776 |
| [33] | Pinski F J, Simpson G, Stuart A M and Weber H 2015 Algorithms for Kullback-Leibler approximation of probability measures in infinite dimensions SIAM J. Sci. Comput.37 A2733-57 · Zbl 1348.60033 · doi:10.1137/14098171x |
| [34] | Pinski F J, Simpson G, Stuart A M and Weber H 2015 Kullback-Leibler approximation for probability measures on infinite dimensional spaces SIAM J. Math. Anal.47 4091-122 · Zbl 1342.60049 · doi:10.1137/140962802 |
| [35] | Arridge S, Maass P, Öktem O and Schönlieb C-B 2019 Solving inverse problems using data-driven models Acta Numer.28 1-174 · Zbl 1429.65116 · doi:10.1017/s0962492919000059 |
| [36] | Nawaz M A 2019 Efficient probabilistic inversion of geophysical data PhD Thesis |
| [37] | Lassas M and Siltanen S 2004 Can one use total variation prior for edge-preserving Bayesian inversion? Inverse Problems20 1537-63 · Zbl 1062.62260 · doi:10.1088/0266-5611/20/5/013 |
| [38] | Kaipio J and Somersalo E 2007 Statistical inverse problems: discretization, model reduction and inverse crimes J. Comput. Appl. Math.198 493-504 · Zbl 1101.65008 · doi:10.1016/j.cam.2005.09.027 |
| [39] | Natterer F 2001 The Mathematics of Computerized Tomography(Classics in Applied Mathematics) (Philadelphia, PA: SIAM: Society for Industrial and Applied Mathematics) · Zbl 0973.92020 · doi:10.1137/1.9780898719284 |
| [40] | Tarantola A 2005 Inverse Problem Theory and Methods for Model Parameter Estimation (Philadelphia, PA: Society for Industrial and Applied Mathematics) · Zbl 1074.65013 · doi:10.1137/1.9780898717921 |
| [41] | Li D, Svensson J, Thomsen H, Medina F, Werner A and Wolf R 2013 Bayesian soft x-ray tomography using non-stationary Gaussian processes Rev. Sci. Instrum.84 083506 · doi:10.1063/1.4817591 |
| [42] | Plagemann C, Kersting K and Burgard W 2008 Nonstationary Gaussian process regression using point estimates of local smoothness Machine Learning and Knowledge Discovery in Databases European Conference, ECML PKDD 2008 Proc., Part II(Antwerp, Belgium, 15-19 September 2008) · doi:10.1007/978-3-540-87481-2_14 |
| [43] | Purisha Z, Jidling C, Wahlström N, Schön T B and Särkkä S 2019 Probabilistic approach to limited-data computed tomography reconstruction Inverse Problems35 105004 · Zbl 1480.94008 · doi:10.1088/1361-6420/ab2e2a |
| [44] | Frese T, Bouman C A and Sauer K 2002 Adaptive wavelet graph model for Bayesian tomographic reconstruction IEEE Trans. Image Process.11 756-70 · doi:10.1109/tip.2002.801586 |
| [45] | Chen Y, Li Y, Yu W, Luo L, Chen W and Toumoulin C 2011 Joint-MAP tomographic reconstruction with patch similarity based mixture prior model Multiscale Model. Simul.9 1399-419 · Zbl 1244.94005 · doi:10.1137/100814184 |
| [46] | Antun V, Renna F, Poon C, Adcock B and Hansen A C 2020 On instabilities of deep learning in image reconstruction and the potential costs of AI Proc. Natl Acad. Sci. USA 201907377 · doi:10.1073/pnas.1907377117 |
| [47] | Evans L C 2014 An Introduction to Stochastic Differential Equations (Providence, RI: American Mathematical Society) |
| [48] | Lasanen S, Roininen L and Huttunen J M J 2018 Elliptic boundary value problems with Gaussian white noise loads Stoch. Process. their Appl.128 3607-27 · Zbl 1408.60051 · doi:10.1016/j.spa.2017.11.007 |
| [49] | Brenner S C and Scott L R 2008 The Mathematical Theory of Finite Element Methods(Texts in Applied Mathematics, vol 5) 3rd edn (New York: Springer) · Zbl 1135.65042 · doi:10.1007/978-0-387-75934-0 |
| [50] | Bogachev V I 2018 Weak Convergence of Measures(Mathematical Surveys and Monographs, vol 34) (Providence, RI: American Mathematical Society) · Zbl 1412.60003 · doi:10.1090/surv/234 |
| [51] | Kallenberg O 2002 Foundations of Modern Probability(Probability and its Applications (New York)) 2nd edn (New York: Springer) · Zbl 0996.60001 · doi:10.1007/978-1-4757-4015-8 |
| [52] | Rudin W 1976 Principles of Mathematical Analysis(International Series in Pure and Applied Mathematics, vol 39) (New York: McGraw-Hill) · Zbl 0346.26002 |
| [53] | Marcus M B and Rosen J 2006 Markov Processes, Gaussian Processes, and Local Times(Cambridge Studies in Advanced Mathematics, vol 100) (Cambridge: Cambridge University Press) · Zbl 1129.60002 · doi:10.1017/CBO9780511617997 |
| [54] | Bogachev V I 1998 Gaussian Measures(Mathematical Surveys and Monographs, vol 2) (Providence, RI: American Mathematical Society) · Zbl 0913.60035 · doi:10.1090/surv/062 |
| [55] | Lasanen S 2012 Non-Gaussian statistical inverse problems. Part II: posterior convergence for approximated unknowns Inverse Problems Imaging6 267-87 · Zbl 1263.62042 · doi:10.3934/ipi.2012.6.267 |
| [56] | Dong K, Eriksson D, Nickisch H, Bindel D and Wilson A G 2017 Scalable log determinants for Gaussian process kernel learning Advances in Neural Information Processing Systems 6327-37 |
| [57] | Yu Y and Meng X-L 2011 To center or not to center: that is not the question- an ancillarity-sufficiency interweaving strategy (ASIS) for boosting MCMC efficiency J. Comp. & Graph. Stat.20 531-70 · doi:10.1198/jcgs.2011.203main |
| [58] | Gamerman H F L D 2006 Markov Chain Monte Carlo (London: Taylor & Francis Inc) · doi:10.1201/9781482296426 |
| [59] | Wendland H 2017 Numerical Linear Algebra (Cambridge: Cambridge University Press) · Zbl 1386.65004 · doi:10.1017/9781316544938 |
| [60] | Cui T, Law K J H and Marzouk Y M jan 2016 Dimension-independent likelihood-informed MCMC J. Comput. Phys.304 109-37 · Zbl 1349.65009 · doi:10.1016/j.jcp.2015.10.008 |
| [61] | Buzug T 2008 Computed Tomography (Berlin: Springer GmbH) |
| [62] | Kak A C and Slaney M 2001 Principles of Computerized Tomographic Imaging (Philadelphia, PA: Society for Industrial and Applied Mathematics) · Zbl 0984.92017 · doi:10.1137/1.9780898719277 |
| [63] | Deans S R 1983 The Radon Transform and Some of its Applications (New York: Wiley) · Zbl 0561.44001 |
| [64] | Okuta R, Unno Y, Nishino D, Hido S and Loomis C 2017 CuPy: a NumPy-compatible library for NVIDIA GPU calculations Proc. MM ’17 Proc. of the 25th ACM Int. Conf. on Multimedia 1217-20 |
| [65] | van der Walt S, Schönberger J L, Nunez-Iglesias J, Boulogne F, Warner J D, Yager N, Gouillart E and Yu T 2014 Scikit-image: image processing in Python PeerJ2 e453 · doi:10.7717/peerj.453 |
| [66] | Mueller J and Siltanen S 2012 Linear and Nonlinear Inverse Problems with Practical Applications (Philadelphia: Society for Industrial and Applied Mathematics) · Zbl 1262.65124 · doi:10.1137/1.9781611972344 |
| [67] | Suuronen J, Emzir M, Lasanen S, Särkkä S and Roininen L 2020 Enhancing industrial X-ray tomography by data-centric statistical methods Data-Centric Eng.1 e10 · doi:10.1017/dce.2020.10 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
