×
Lan, Shiwei; Kang, Lulu

Sampling constrained continuous probability distributions: a review. (English) Zbl 07910655

Wiley Interdiscip. Rev., WIREs Comput. Stat. 15, No. 6, Article ID e1608, 18 p. (2023).

MSC:

62-08 Computational methods for problems pertaining to statistics

Keywords:

constrained sampling; Hamilton Monte Carlo; regularized regression; Riemannian Monte Carlo; truncated multivariate Gaussian

Software:

tmg; CoMinED
Cite Review PDF
Full Text: DOI arXiv

References:

[1] Ahn, K., & Chewi, S. (2021). Efficient constrained sampling via the mirror‐langevin algorithm. Advances in Neural Information Processing Systems, 34, 28405-28418.
[2] Amzal, B., Bois, F. Y., Parent, E., & Robert, C. P. (2006). Bayesian‐optimal design via interacting particle systems. Journal of the American Statistical Association, 101(474), 773-785. · Zbl 1119.62308
[3] Andersen, H. C. (1983). Rattle: A “velocity” version of the shake algorithm for molecular dynamics calculations. Journal of Computational Physics, 52(1), 24-34. · Zbl 0513.65052
[4] Betancourt, M. (2011). Nested sampling with constrained hamiltonian Monte Carlo. AIP Conference Proceedings, 1305, 165-172.
[5] Bierkens, J., Fearnhead, P., & Roberts, G. (2019). The zig‐zag process and super‐efficient sampling for bayesian analysis of big data. The Annals of Statistics, 47(3), 1288-1320. · Zbl 1417.65008
[6] Brubaker, M. A., Salzmann, M., & Urtasun, R. (2012). A family of mcmc methods on implicitly defined manifolds. In N. D.Lawrence (ed.) & M. A.Girolami (ed.) (Eds.), Proceedings of the fifteenth international conference on artificial intelligence and statistics (AISTATS‐12) (pp. 161-172). ACM.
[7] Byrne, S., & Girolami, M. (2013). Geodesic Monte Carlo on embedded manifolds. Scandinavian Journal of Statistics, 40(4), 825-845. · Zbl 1349.62186
[8] Chaudhry, S., Lautzenheiser, D., & Ghosh, K. (2021). An efficient scheme for sampling in constrained domains. arXiv Preprints, arXiv:2110.10840. https://doi.org/10.48550/ARXIV.2110.10840 · doi:10.48550/ARXIV.2110.10840
[9] Chewi, S., Gerber, P. R., Lu, C., Le Gouic, T., & Rigollet, P. (2022). Rejection sampling from shape‐constrained distributions in sublinear time. In International conference on artificial intelligence and statistics, pp. 2249-2265.
[10] Das, N., & Bhattacharya, R. (2020). Optimal transport based filtering with nonlinear state equality constraints. IFAC‐PapersOnLine, 53(2), 2373-2378.
[11] Draguljić, D., Santner, T. J., & Dean, A. M. (2012). Noncollapsing space‐filling designs for bounded nonrectangular regions. Technometrics, 54(2), 169-178.
[12] Drovandi, C. C., & Pettitt, A. N. (2013). Bayesian experimental design for models with intractable likelihoods. Biometrics, 69(4), 937-948. · Zbl 1419.62182
[13] Drovandi, C. C., & Tran, M.‐N. (2018). Improving the efficiency of fully bayesian optimal design of experiments using randomised quasi‐Monte Carlo. Bayesian Analysis, 13(1), 139-162. · Zbl 06873721
[14] Frank, L. E., & Friedman, J. H. (1993). A statistical view of some chemometrics regression tools. Technometrics, 35(2), 109-135. · Zbl 0775.62288
[15] Fu, W. J. (1998). Penalized regressions: The bridge versus the lasso. Journal of Computational and Graphical Statistics, 7(3), 397-416.
[16] Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., & Rubin, D. B. (2013). Bayesian data analysis. Chapman and Hall/CRC.
[17] Gessner, A., Kanjilal, O., & Hennig, P. (2020). Integrals over Gaussians under linear domain constraints. In International conference on artificial intelligence and statistics, pages 2764-2774.
[18] Girolami, M., & Calderhead, B. (2011). Riemann manifold Langevin and Hamiltonian Monte Carlo methods. Journal of the Royal Statistical Society, Series B, 73(2), 123-214. · Zbl 1411.62071
[19] Hans, C. (2009). Bayesian lasso regression. Biometrika, 96(4), 835-845. · Zbl 1179.62038
[20] Held, L., & Holmes, C. C. (2006). Bayesian auxiliary variable models for binary and multinomial regression. Bayesian Analysis, 1(1), 145-168. · Zbl 1331.62142
[21] Holbrook, A., Lan, S., Streets, J., & Shahbaba, B. (2020). Nonparametric fisher geometry with application to density estimation. In J.Peters (ed.) & D.Sontag (ed.) (Eds.), Proceedings of the 36th conference on uncertainty in artificial intelligence (UAI), volume 124 of proceedings of machine learning research (pp. 101-110). PMLR.
[22] Holbrook, A., Lan, S., Vandenberg‐Rodes, A., & Shahbaba, B. (2018). Geodesic lagrangian Monte Carlo over the space of positive definite matrices: With application to bayesian spectral density estimation. Journal of Statistical Computation and Simulation, 88(5), 982-1002. · Zbl 07192588
[23] Huang, C., Joseph, V. R., & Ray, D. M. (2021). Constrained minimum energy designs. Statistics and Computing, 31(6), 1-15. · Zbl 1477.62009
[24] Iserles, A. (1986). Generalized leapfrog methods. IMA Journal of Numerical Analysis, 6(4), 381-392. · Zbl 0637.65089
[25] Jacob, P. E., Gong, R., Edlefsen, P. T., & Dempster, A. P. (2021). A gibbs sampler for a class of random convex polytopes. Journal of the American Statistical Association, 116(535), 1181-1192. · Zbl 1510.62140
[26] Jessen, R. J. (1970). Probability sampling with marginal constraints. Journal of the American Statistical Association, 65(330), 776-796. · Zbl 0195.48504
[27] Kang, L. (2019). Stochastic coordinate‐exchange optimal designs with complex constraints. Quality Engineering, 31(3), 401-416.
[28] Kang, L., Roshan Joseph, V., & Brenneman, W. A. (2011). Design and modeling strategies for mixture‐of‐mixtures experiments. Technometrics, 53(2), 125-136.
[29] Kang, X., Ranganathan, S., Kang, L., Gohlke, J., & Deng, X. (2021). Bayesian auxiliary variable model for birth records data with qualitative and quantitative responses. Journal of Statistical Computation and Simulation, 91(16), 3283-3303. · Zbl 07497664
[30] Kook, Y., Lee, Y. T., Shen, R., & Vempala, S. S. (2022). Sampling with riemannian Hamiltonian Monte Carlo in a constrained space. arXiv Preprints, arXiv:2202.01908. https://doi.org/10.48550/ARXIV.2202.01908 · doi:10.48550/ARXIV.2202.01908
[31] Lan, S., Holbrook, A., Elias, G. A., Fortin, N. J., Ombao, H., & Shahbaba, B. (2020). Flexible Bayesian dynamic modeling of correlation and covariance matrices. Bayesian Analysis, 15(4), 1199-1228. · Zbl 1480.62100
[32] Lan, S., & Shahbaba, B. (2016). Algorithmic advances in Riemannian geometry and applications, chapter 2‐sampling constrained probability distributions using spherical augmentation. In S. B.Kang (ed.) (Ed.), Advances in computer vision and pattern recognition (1st ed., pp. 25-71). Springer International Publishing. · Zbl 1395.62054
[33] Lan, S., Stathopoulos, V., Shahbaba, B., & Girolami, M. (2015). Markov chain Monte Carlo from lagrangian dynamics. Journal of Computational and Graphical Statistics, 24(2), 357-378.
[34] Lan, S., Zhou, B., & Shahbaba, B. (2014). Spherical hamiltonian Monte Carlo for constrained target distributions. In E. P.Xing (ed.) (Ed.), The 31st international conference on machine learning (pp. 629-637). PMLR.
[35] Lang, L., Shiang Chen, W., Bakshi, B. R., Goel, P. K., & Ungarala, S. (2007). Bayesian estimation via sequential Monte Carlo sampling—Constrained dynamic systems. Automatica, 43(9), 1615-1622. · Zbl 1128.93387
[36] Legrain, G. (2021). Non‐negative moment fitting quadrature rules for fictitious domain methods. Computers & Mathematics with Applications, 99, 270-291. · Zbl 1524.74427
[37] Leimkuhler, B., & Reich, S. (1994). Symplectic integration of constrained hamiltonian systems. Mathematics of Computation, 63(208), 589. · Zbl 0813.65103
[38] Leobacher, G., & Pillichshammer, F. (2014). Introduction to quasi‐Monte Carlo integration and applications. Springer. · Zbl 1309.65006
[39] Li, Y., & Ghosh, S. K. (2015). Efficient sampling methods for truncated multivariate normal and student‐t distributions subject to linear inequality constraints. Journal of Statistical Theory and Practice, 9(4), 712-732. · Zbl 1423.62047
[40] Neal, P., & Roberts, G. O. (2008). Optimal scaling for random walk metropolis on spherically constrained target densities. Methodology and Computing in Applied Probability, 10, 277-297. · Zbl 1141.60316
[41] Neal, P., Roberts, G. O., & Yuen, W. K. (2012). Optimal scaling of random walk metropolis algorithms with discontinuous target densities. Annals of Applied Probability, 22(5), 1880-1927. · Zbl 1259.60082
[42] Neal, R. M. (1994). An improved acceptance procedure for the hybrid Monte Carlo algorithm. Journal of Computational Physics, 111(1), 194-203. · Zbl 0797.65115
[43] Neal, R. M. (2011). MCMC using Hamiltonian dynamics. In S.Brooks (ed.), A.Gelman (ed.), G.Jones (ed.), & X. L.Meng (ed.) (Eds.), Handbook of Markov chain Monte Carlo (pp. 113-162). Chapman and Hall/CRC. · Zbl 1229.65018
[44] Nishimura, A., Zhang, Z., & Suchard, M. A. (2021). Hamiltonian zigzag sampler got more momentum than its markovian counterpart: Equivalence of two zigzags under a momentum refreshment limit. arXiv Preprints, arXiv:2104.07694.
[45] Olander, J. (2020). Constrained space mcmc methods for nested sampling Bayesian computations (Master’s thesis). Chalmers University of Technology, Gothenburg, Sweden.
[46] Olshanskii, M. A., & Safin, D. (2016). Numerical integration over implicitly defined domains for higher order unfitted finite element methods. Lobachevskii Journal of Mathematics, 37(5), 582-596. · Zbl 1354.65252
[47] Owen, A. B. (2013). Monte Carlo theory, methods and examples. Springer.
[48] Pakman, A., & Paninski, L. (2014). Exact hamiltonian Monte Carlo for truncated multivariate gaussians. Journal of Computational and Graphical Statistics, 23(2), 518-542.
[49] Park, T., & Casella, G. (2008). The bayesian lasso. Journal of the American Statistical Association, 103(482), 681-686. · Zbl 1330.62292
[50] Piepel, G., Cooley, S., & Jones, B. (2005). Construction of a 21‐component layered mixture experiment design using a new mixture coordinate‐exchange algorithm. Quality Engineering, 17(4), 579-594.
[51] Pratola, M. T., Harari, O., Bingham, D., & Flowers, G. E. (2017). Design and analysis of experiments on nonconvex regions. Technometrics, 59(1), 36-47.
[52] Ryan, E. G., Drovandi, C. C., McGree, J. M., & Pettitt, A. N. (2016). A review of modern computational algorithms for bayesian optimal design. International Statistical Review, 84(1), 128-154. · Zbl 07763475
[53] Saye, R. I. (2022). High‐order quadrature on multi‐component domains implicitly defined by multivariate polynomials. Journal of Computational Physics, 448, 110720. · Zbl 1537.65022
[54] Shen, S., Kang, L., & Deng, X. (2020). Additive heredity model for the analysis of mixtureof‐ mixtures experiments. Technometrics, 62(2), 265-276.
[55] Sherlock, C., & Roberts, G. O. (2009). Optimal scaling of the random walk metropolis on elliptically symmetric unimodal targets. Bernoulli, 15(3), 774-798. · Zbl 1215.60047
[56] Skilling, J. (2006). Nested sampling for general Bayesian computation. Bayesian Analysis, 1(4), 833-859. · Zbl 1332.62374
[57] Stinstra, E., denHertog, D., Stehouwer, P., & Vestjens, A. (2003). Constrained maximin designs for computer experiments. Technometrics, 45(4), 340-346.
[58] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1), 267-288. · Zbl 0850.62538
[59] Verlet, L. (1967). Computer “experiments” on classical fluids. I. Thermodynamical properties of Lennard‐Jones molecules. Physics Review, 159(1), 98-103.
[60] Wang, L. (2008). Karhunen‐Loeve expansions and their applications (PhD thesis). London School of Economics and Political Science, UK.
[61] Zhang, Z., Nishimura, A., Bastide, P., Ji, X., Payne, R. P., Goulder, P., Lemey, P., & Suchard, M. A. (2021). Large‐scale inference of correlation among mixed‐type biological traits with phylogenetic multivariate probit models. The Annals of Applied Statistics, 15(1), 230-251. · Zbl 1475.62273
[62] Zhang, Z., Nishimura, A., trov˜ao, N. S., Cherry, J. L., Holbrook, A. J., Ji, X., Lemey, P., & Suchard, M. A. (2022). Accelerating Bayesian inference of dependency between complex biological traits. arXiv Preprints, arXiv:2201.07291.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.