×

Weighted particle tempering. (English) Zbl 1464.62037

Summary: The application of Bayesian methods often requires Metropolis-Hastings or related algorithms to sample from an intractable posterior distribution. In especially challenging cases, such as with strongly correlated parameters or multimodal posteriors, exotic forms of Metropolis-Hastings are preferred for generating samples within a reasonable time. These algorithms require nontrivial and often prohibitive tuning, with little or no performance guarantees. In light of this difficulty, a new, parallelizable algorithm called weighted particle tempering is introduced. Weighted particle tempering is easily tuned and suitable for a broad range of applications. The algorithm works by running multiple random walk Metropolis chains directed at a tempered version of the target distribution, weighting the iterates and resampling. The algorithm’s performance monotonically improves with more of these underlying chains, a feature that simplifies tuning. Through the use of simulation studies, weighted particle tempering is shown to outperform two similar methods: parallel tempering and parallel hierarchical sampling. In addition, two case studies are explored: breast cancer classification and graphical models for financial data.

MSC:

62-08 Computational methods for problems pertaining to statistics

Software:

HdBCS; UCI-ml
Full Text: DOI

References:

[1] Chipman, H. A.; George, E. I.; McCulloch, R. E., Bayesian CART model search, J. Amer. Statist. Assoc., 93, 443, 935-948, (1998)
[2] Geman, S.; Geman, D., Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images, IEEE Trans. Pattern Anal. Mach. Intell., 84, 6, 721-741, (1984) · Zbl 0573.62030
[3] Geyer, C.J., 1991. Markov Chain Monte Carlo maximum likelihood, In: Computing Science and Statistics: Proceedings of the 23rd Symposium Interface, pp.156-163.
[4] Geyer, C. J.; Thompson, E. A., Annealing Markov chain Monte Carlo with applications to ancestral inference, J. Amer. Statist. Assoc., 90, 431, 909-920, (1995) · Zbl 0850.62834
[5] Giudici, P.; Green, P. J., Decomposable graphical Gaussian model determination, Biometrika, 86, 4, 785-801, (1999) · Zbl 0940.62019
[6] Gramacy, R.; Samworth, R.; King, R., Importance tempering, Stat. Comput., 20, 1, 1-7, (2010)
[7] Jones, B.; Carvalho, C.; Dobra, A.; Hans, C.; Carter, C.; West, M., Experiments in stochastic computation for high-dimensional graphical models, Statist. Sci., 20, 4, 388-400, (2005) · Zbl 1130.62408
[8] Kim, H. J.; MacEachern, S. N., The generalized multiset sampler, J. Comput. Graph. Statist., 24, 4, 1134-1154, (2015)
[9] Kone, A.; Kofke, D. A., Selection of temperature intervals for parallel-tempering simulations, J. Chem. Phys., 122, 20, (2005)
[10] Kou, S. C.; Zhou, Q.; Wong, W. H., Equi-energy sampler with applications in satistical inference and statistical mechanics, Ann. Statist., 34, 4, 1581-1619, (2006) · Zbl 1246.82054
[11] Lauritzen, S. L., Graphical models, (1996), Oxford University Press · Zbl 0907.62001
[12] Leman, S. C.; Chen, Y.; Lavine, M., The multiset sampler, J. Amer. Statist. Assoc., 104, 487, 1029-1041, (2009) · Zbl 1388.60133
[13] Liang, F.; Wong, W. H., Real-parameter evolutionary Monte Carlo with applications to Bayesian mixture models, J. Amer. Statist. Assoc., 96, 454, 653-666, (2001) · Zbl 1017.62022
[14] Lichman, M., 2013. UCI Machine Learning Repository. URL http://archive.ics.uci.edu/ml.
[15] Liu, J. S.; Liang, F.; Wong, W. H., The multiple-try method and local optimization in metropolis sampling, J. Amer. Statist. Assoc., 95, 449, 121-134, (2000) · Zbl 1072.65505
[16] Marinari, E.; Parisi, G., Simulated tempering: A new Monte Carlo scheme, Europhys. Lett., 19, 6, 451-458, (1992)
[17] Metropolis, N.; Rosenbluth, A. W.; Rosenbluth, M. N.; Teller, A. H.; Teller, E., Equation of state calculations by fast computing machines, J. Chem. Phys., 21, 6, 1087-1092, (1953) · Zbl 1431.65006
[18] Miasojedow, B.; Moulines, E.; Vihola, M., An adaptive parallel tempering algorithm, J. Comput. Graph. Statist., 22, 3, 649-664, (2013)
[19] Neal, R. M., Sampling from multimodal distributions using tempered transitions, Stat. Comput., 6, 4, 353-366, (1996)
[20] Rigat, F.; Mira, A., Parallel hierarchical sampling: A general-purpose interacting Markov chains Monte Carlo algorithm, Comput. Statist. Data Anal., 56, 6, 1450-1467, (2012) · Zbl 1246.65026
[21] Swendsen, R. H.; Wang, J.-S., Replica Monte Carlo simulation of spin-glasses, Phys. Rev. Lett., 57, 21, 2607-2609, (1986)
[22] Tierney, L., Markov chains for exploring posterior distributions, Ann. Statist., 22, 4, 1701-1728, (1994) · Zbl 0829.62080
[23] Wolberg, W. H.; Mangasarian, O. L., Multisurface method of pattern separation for medical diagnosis applied to breast cytology, Proc. Natl. Acad. Sci., 87, 23, 9193-9196, (1990) · Zbl 0709.92537
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.