×

Fast sampling in a linear-Gaussian inverse problem. (English) Zbl 1398.94081

Summary: We solve the inverse problem of deblurring a pixelized image of Jupiter using regularized deconvolution and by sample-based Bayesian inference. By efficiently sampling the marginal posterior distribution for hyperparameters, then the full conditional for the deblurred image, we find that we can evaluate the posterior mean faster than regularized inversion, when selection of the regularizing parameter is considered. To our knowledge, this is the first demonstration of sampling and inference being computed in less time than regularized inversion, in a significant inverse problem. Comparison to random-walk Metropolis-Hastings and block Gibbs Markov chain Monte Carlo shows that marginal then conditional sampling also outperforms these more common sampling algorithms. The asymptotic cost of an independent sample is one linear solve, implying that, when problem-specific computations are feasible, sample-based Bayesian inference may be performed directly over function spaces when that limit exists.

MSC:

94A20 Sampling theory in information and communication theory
45Q05 Inverse problems for integral equations
47N30 Applications of operator theory in probability theory and statistics
62F15 Bayesian inference
65C60 Computational problems in statistics (MSC2010)

References:

[1] F. Acosta, M. L. Huber, and G. L. Jones, {\it Markov Chain Monte Carlo with Linchpin Variables}. (2015).
[2] S. Agapiou, J. M. Bardsley, O. Papaspiliopoulos, and A. M. Stuart, {\it Analysis of the Gibbs sampler for hierarchical inverse problems}, SIAM/ASA J. Uncertain. Quantif., 2 (2014), pp. 511-544. · Zbl 1308.62097
[3] S. Banerjee, B. P. Carlin, and A. E. Gelfand, {\it Hierarchical Modeling and Analysis for Spatial Data}, Chapman and Hall/CRC, Boca Raton, FL, 2004. · Zbl 1053.62105
[4] J. M. Bardsley, {\it MCMC-based image reconstruction with uncertainty quantification}, SIAM J. Sci. Comput., 34 (2012), pp. A1316-A1332. · Zbl 1246.15022
[5] J. O. Berger, J. M. Bernardo, and D. Sun, {\it Overall objective priors}, Bayesian Anal., 10 (2015), 189221. · Zbl 1335.62039
[6] D. Colton and R. Kress, {\it Inverse Acoustic and Electromagnetic Scattering Theory}, 2nd ed. Springer-Verlag, Berlin, 1998. · Zbl 0893.35138
[7] G. Dahlquist and \AA. Björck, {\it Numerical Methods}, Prentice-Hall, Englewood Cliffs, NJ, 1974. · Zbl 1153.65001
[8] W. Dȩbski, {\it Seismic tomography by Monte Carlo sampling}, Pure Appl. Geophys., 167 (2010), pp. 131-152, .
[9] C. Fox and G. K. Nicholls, {\it Sampling conductivity images via MCMC}, in The Art and Science of Bayesian Image Analysis, K. V. Mardia, C. A. Gill, and R. G. Aykroyd, eds., University of Leeds, Leeds, England, 1997, pp. 91-100.
[10] C. J. Geyer, {\it Practical Markov chain Monte Carlo}, Statist. Sci., 7, (1992), pp. 473-483.
[11] C. Gilavert, S. Moussaoui, and J. Idier, {\it Efficient Gaussian sampling for solving large-scale inverse problems using MCMC}, IEEE Trans. Signal Process., 63 (2015), pp. 70-80, . · Zbl 1394.94822
[12] W. Gilks, S. Richardson, and D. Spiegelhalter, {\it Introducing Markov chain Monte Carlo}, in Markov Chain Monte Carlo in Practice, W. Gilks, S. Richardson, and D. Spiegelhalter, eds., Chapman and Hall, Boca Raton, FL, 1996, pp. 1-19. · Zbl 0845.60072
[13] I. Gohberg, S. Goldberg, and N. Krupnik, {\it Traces and Determinants of Linear Operators}, Oper. Theory Adv. Appl. 110, Springer, Basel, 2000. · Zbl 0946.47013
[14] U. Grenander and M. Miller, {\it Pattern Theory: From Representation to Inference}, Oxford University Press, New York, 2007. · Zbl 1259.62089
[15] P. C. Hansen, {\it Rank-deficient and Discrete Ill-posed Problems: Numerical Aspects of Linear Inversion}, SIAM, Philadelphia, 1998.
[16] P. C. Hansen, {\it Regularization tools: A Matlab package for analysis and solution of discrete ill-posed problems}, Numer. Algorithms, 6 (1994), pp. 1-35, . · Zbl 0789.65029
[17] P. C. Hansen and D. P. O’Leary, {\it The use of the L-curve in the regularization of discrete ill-posed problems}, SIAM J. Sci. Comput., 14 (1993), pp. 1487-1503, . · Zbl 0789.65030
[18] D. Higdon, {\it A primer on space-time modelling from a Bayesian perspective}, in Statistical Methods of Spatio-Temporal Systems, B. Finkenstadt, L. Held, and V. Isham, eds., Chapman and Hall/CRC, Boca Raton, FL, 2006, pp. 217-279. · Zbl 1121.62081
[19] M. A. Hurn, O. Husby, and H. Rue, {\it Advances in Bayesian image analysis}, in Highly Structured Stochastic Systems, P. J. Green, N. Hjort, and S. Richardson, eds., Oxford University Press, Oxford, 2003, pp. 302-322. · Zbl 1044.62110
[20] H. Jeffreys, {\it An invariant form for the prior probability in estimation problems}, Proc. the R. Soc. Lond. A Math. Phys. Eng. Sci., 186 (1946), pp. 453-461, . · Zbl 0063.03050
[21] C. Kipnis and S. R. S. Varadhan, {\it Central limit theorem for additive functionals of reversible Markov processes and applications to simple exclusions}, Comm. Math. Phys., 104 (1986), pp. 1-19. · Zbl 0588.60058
[22] G. Meurant, {\it Estimates of the trace of the inverse of a symmetric matrix using the modified Chebyshev algorithm}, Numer. Algorithms, 51 (2009), pp. 309-318, . · Zbl 1166.65335
[23] O. Nevanlinna, {\it Convergence of Iterations for Linear Equations}, Lectures in Math. Birkhäuser, Basel, 1993. · Zbl 0846.47008
[24] G. Nicholls and M. Jones, {\it Radiocarbon dating with temporal order constraints}, J. R. Stat. Soc. Ser. C Appl. Stat., 50 (2001), pp. 503-521. · Zbl 1112.62319
[25] R. A. Norton and C. Fox, {\it Tuning of MCMC with Langevin, Hamiltonian, and other stochastic autoregressive proposals}, arXiv:1610.00781 [stat.CO], 2016.
[26] D. S. Oliver, N. He, and A. C. Reynolds, {\it Conditioning permeability fields to pressure data}, in ECMOR V-5th European Conference on the Mathematics of Oil Recovery, Mining University Leoben, Leoben, pp. 259-270, 1996.
[27] F. Orieux, O. Féron, and J.-F. Giovannelli, {\it Sampling high-dimensional Gaussian distributions for general linear inverse problems.}, IEEE Signal Process. Lett., 19 (2012), pp. 251-254.
[28] O. Papaspiliopoulos and G. O. Roberts, {\it Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models}, Biometrika, 95 (2008), pp. 169-186, . · Zbl 1437.62576
[29] O. Papaspiliopoulos, G. O. Roberts, and M. Sköld, {\it A general framework for the parametrization of hierarchical models}, Statist. Sci., 22 (2007), pp. 59-73, . · Zbl 1246.62195
[30] W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling, {\it Numerical Recipes: The Art of Scientific Computing}, Cambridge University Press, New York, 1986. · Zbl 0587.65003
[31] C. P. Robert, N. Chopin, and J. Rousseau, {\it Harold Jeffreys’s theory of probability revisited}, Stat. Sci., 24 (2009), pp. 141-172. · Zbl 1328.62012
[32] G. Roberts, {\it ST911 Fundamentals of Statistical Inference. Part III}, (2015).
[33] H. Rue and L. Held, {\it Gaussian Markov Random Fields: Theory and Applications}, Chapman Hall, New York, 2005. · Zbl 1093.60003
[34] D. Simpson, F. Lindgren, and H. Rue, {\it Think continuous: Markovian Gaussian models in spatial statistics}, Spat. Stat., 1 (2012), p. 16-–29.
[35] A. D. Sokal, {\it Monte Carlo methods in statistical mechanics: Foundations and new algorithms}, in “Functional Integration: Basics and Applications”, NATO ASI Ser. B. 361, Plenum, New York, 1997, pp. 131-192. 1996. · Zbl 0890.65006
[36] D. A. van Dyk, {\it Marginal Markov chain Monte Carlo methods}, Statist. Sinica, 20 (2010), pp. 1423-1454. · Zbl 1410.60073
[37] C. Vogel, {\it Computational Methods for Inverse Problems}, Frontiers Appl. Math 23, SIAM, Philadelphia, 2002. · Zbl 1008.65103
[38] D. Watzenig and C. Fox, {\it A review of statistical modelling and inference for electrical capacitance tomography}, Meas. Sci. Technol., 20(2009), 052002, .
[39] C. K. Wikle, R. F. Milliff, D. Nychka, and L. M. Berliner, {\it Spatiotemporal hierarchical Bayesian modeling: Tropical ocean surface winds}, J. Amer. Statist. Assoc., 96 (2001), pp. 382-397, . · Zbl 1022.62117
[40] U. Wolff, {\it Monte Carlo errors with less errors}, Comput. Phys. Commun., 156 (2004), pp. 143-153. · Zbl 1196.65138
[41] N. Young, {\it An Introduction to Hilbert Space}, Cambridge University Press, Cambridge, 1988. · Zbl 0645.46024
[42] R. Zhang, C. Czado, and K. Sigloch, {\it A Bayesian linear model for the high-dimensional inverse problem of seismic tomography}, Ann. Appl. Stat., 7 (2013), pp. 1111-1138. · Zbl 1288.62045
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.