×

Markov chain Monte Carlo and irreversibility. (English) Zbl 1387.60116

Summary: Markov chain Monte Carlo (MCMC) methods are statistical methods designed to sample from a given measure \(\pi\) by constructing a Markov chain that has \(\pi\) as invariant measure and that converges to \(\pi\). Most MCMC algorithms make use of chains that satisfy the detailed balance condition with respect to \(\pi\); such chains are therefore reversible. On the other hand, recent work [C.-R. Hwang et al., Ann. Appl. Probab. 15, No. 2, 1433–1444 (2005; Zbl 1069.60065); T. Lelièvre et al., J. Stat. Phys. 152, No. 2, 237–274 (2013; Zbl 1276.82042); L. Rey-Bellet and K. Spiliopoulos, Nonlinearity 28, No. 7, 2081–2103 (2015; Zbl 1338.60086); Electron. Commun. Probab. 20, Paper No. 15, 16 p. (2015; Zbl 1347.60015)] has stressed several advantages of using irreversible processes for sampling. Roughly speaking, irreversible diffusions converge to equilibrium faster (and lead to smaller asymptotic variance as well). In this paper we discuss some of the recent progress in the study of nonreversible MCMC methods. In particular: i) we explain some of the difficulties that arise in the analysis of nonreversible processes and we discuss some analytical methods to approach the study of continuous-time irreversible diffusions; ii) most of the rigorous results on irreversible diffusions are available for continuous-time processes; however, for computational purposes one needs to discretize such dynamics. It is well known that the resulting discretized chain will not, in general, retain all the good properties of the process that it is obtained from. In particular, if we want to preserve the invariance of the target measure, the chain might no longer be reversible. Therefore iii) we conclude by presenting an MCMC algorithm, the SOL-HMC algorithm [the author et al., Bernoulli 22, No. 1, 60–106 (2016; Zbl 1346.60119)], which results from a nonreversible discretization of a nonreversible dynamics.

MSC:

60J22 Computational methods in Markov chains
60J60 Diffusion processes
65C05 Monte Carlo methods
65C30 Numerical solutions to stochastic differential and integral equations
Full Text: DOI

References:

[2] Baldi, P., Calcolo delle Probabilità e Statistica (1998), Mc Graw-Hill: Mc Graw-Hill Milano
[3] Bakry, D.; Gentil, I.; Ledoux, M., Analysis and Geometry of Markov Diffusion Operators (2014), Springer · Zbl 1376.60002
[4] Beskos, A.; Stuart, A. M., MCMC methods for sampling function space, (ICIAM Invited Lecture (2007), European Mathematical Society: European Mathematical Society Zürich.)
[5] Beskos, A.; Pinski, F.; Sanz-Serna, J. M.; Stuart, A. M., Hybrid Monte-Carlo on Hilbert spaces, Stoch. Proc. Appl., 121, 2201-2230 (2011) · Zbl 1235.60091
[7] Da Prato, G.; Zabczyk, J., Stochastic Equations in Infinite Dimensions (1992), Cambridge University Press: Cambridge University Press Cambridge · Zbl 0761.60052
[8] Diaconis, P.; Holmes, S.; Neal, R. M., Analysis of non-reversible Markov Chain Sampler, Ann. Appl. Prob., 10, No. 3, 726-752 (2000) · Zbl 1083.60516
[11] Diaconis, P., The Markov chain Monte Carlo revolution, Bull. Amer. Math. Soc., 46, 179-205 (2009) · Zbl 1168.60032
[12] Durrett, R., Probability: Theory and Examples, (Cambridge Series in Statistical and Probabilistic Mathematics (2010), Cambridge University Press: Cambridge University Press Cambridge) · Zbl 1202.60001
[13] Gadat, S.; Miclo, L., Spectral decompositions and L2-operator norms of toy hypocoercive semi-groups, Kinet. Relat. Models, 6, no. 2, 317-372 (2013) · Zbl 1262.35134
[14] Gilks, W. R.; Richardson, S.; Spiegelhalter, D. J., Markov Chain Monte Carlo in practice (1996), Chapman and Hall: Chapman and Hall London · Zbl 0832.00018
[17] Horowitz, A. M., A generalized guided Monte Carlo algorithm, Phys. Lett. B, 268, 2, 247-252 (1991)
[19] Kontoyiannis, I.; Meyn, S. P., Geometric ergodicity and the spectral gap of non-reversible Markov chains, Probab. Theory Relat. Fields, 154, 327-339 (2012) · Zbl 1263.60064
[20] Lelièvre, T.; Rousset, M.; Stoltz, G., Free Energy Computations: A Mathematical Perspective (2010), Imperial College Press · Zbl 1227.82002
[21] Lelièvre, T.; Nier, F.; Pavliotis, G. A., Optimal non-reversible linear drift for the convergence to equilibrium of a diffusion, J. Stat. Phys., 152, 237-274 (2013) · Zbl 1276.82042
[22] Meyn, S. P.; Tweedie, R. L., Markov Chains and Stochastic Stability (1993), Communications and Control Engineering Series, Springer London Ltd.: Communications and Control Engineering Series, Springer London Ltd. London · Zbl 0925.60001
[24] Ottobre, M.; Pavliotis, G. A.; Pravda-Starov, K., Exponential return to equilibrium for hypoelliptic quadratic systems, J. Funct. Anal., 262, 4000-4039 (2012) · Zbl 1256.47032
[25] Pavliotis, G., Stochastic Process and Applications (2015), Springer
[26] Neal, R. M., MCMC using Hamiltonian dynamics, Handbook of Markov Chain Monte Carlo, 54, 113-162 (2010) · Zbl 1229.65018
[27] Reed, M.; Simon, B., Methods of Modern Mathematical Physics (1980), Academic Press: Academic Press New York · Zbl 0459.46001
[28] Rey-Bellet, L.; Spiliopoulos, K., Irreversible Langevin samplers and variance reduction: a large deviation approach, Nonlinearity, 28, 2081-2103 (2015) · Zbl 1338.60086
[29] Rey-Bellet, L.; Spiliopoulos, K., Variance reduction for irreversible Langevin samplers and diffusion on graphs, Electronic Communications in Probability, 20, no. 15, 1-16 (2015) · Zbl 1347.60015
[30] Robert, C. P.; Casella, G., Introducing Monte Carlo methods with R. Use R! (2010), Springer: Springer New York · Zbl 1196.65025
[35] Stroock, D. W.; Varadhan, S. R.S., Multidimensional Diffusion Processes (1979), Springer: Springer Berlin · Zbl 0426.60069
[36] Stuart, A. M., Inverse problems: a Bayesian perspective, Acta Numerica, 19, 451-559 (2010) · Zbl 1242.65142
[37] Tierney, L., A note on Metropolis-Hastings kernels for general state spaces, Ann. Appl. Probab., 8 (1998), no. 1, 1-9. · Zbl 0935.60053
[38] Villani, C., Hypocoercivity, Mem. Amer. Math. Soc., 202, 950 (2009) · Zbl 1197.35004
[39] Williams, D., To begin at the beginning:…, (Proc. Sympos., Univ. Durham, Durham, 1980 (1981), Springer: Springer Berlin-New York), 1-55, Lecture Notes in Math., 851 · Zbl 0471.60065
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.