×

MCMC computations for Bayesian mixture models using repulsive point processes. (English) Zbl 07547622

Summary: Repulsive mixture models have recently gained popularity for Bayesian cluster detection. Compared to more traditional mixture models, repulsive mixture models produce a smaller number of well-separated clusters. The most commonly used methods for posterior inference either require to fix a priori the number of components or are based on reversible jump MCMC computation. We present a general framework for mixture models, when the prior of the “cluster centers” is a finite repulsive point process depending on a hyperparameter, specified by a density which may depend on an intractable normalizing constant. By investigating the posterior characterization of this class of mixture models, we derive a MCMC algorithm which avoids the well-known difficulties associated to reversible jump MCMC computation. In particular, we use an ancillary variable method, which eliminates the problem of having intractable normalizing constants in the Hastings ratio. The ancillary variable method relies on a perfect simulation algorithm, and we demonstrate this is fast because the number of components is typically small. In several simulation studies and an application on sociological data, we illustrate the advantage of our new methodology over existing methods, and we compare the use of a determinantal or a repulsive Gibbs point process prior model. Supplementary files for this article are available online.

MSC:

62-XX Statistics

References:

[1] Argiento, R.; Bianchini, I.; Guglielmi, A., “Posterior Sampling From ε-Approximation of Normalized Completely Random Measure Mixtures, Electronic Journal of Statistics, 10, 3516-3547 (2016) · Zbl 1358.62034 · doi:10.1214/16-EJS1168
[2] Argiento, R.; De Iorio, M., Is Infinity That Far? A Bayesian Nonparametric Perspective of Finite Mixture Models, Technical report. arXiv:1904.09733 (2019)
[3] Bardenet, R.; Titsias, M.; Cortes, C.; Lawrence, N. D.; Lee, D. D.; Sugiyama, M.; Garnett, R., Advances in Neural Information Processing Systems, 28, Inference for Determinantal Point Processes Without Spectral Knowledge, 3393-3401 (2015), Curran Associates, Inc
[4] Bianchini, I.; Guglielmi, A.; Quintana, F. A., “Determinantal Point Process Mixtures Via Spectral Density Approach, Bayesian Analysis, 15, 187-214 (2020) · Zbl 1437.62136 · doi:10.1214/19-BA1150
[5] Binder, D. A., “Bayesian Cluster Analysis, Biometrika, 65, 31-38 (1978) · Zbl 0376.62007 · doi:10.1093/biomet/65.1.31
[6] Biscio, C. A. N.; Lavancier, F., “Quantifying Repulsiveness of Determinantal Point Processes, Bernoulli, 22, 2001-2028 (2016) · Zbl 1343.60058 · doi:10.3150/15-BEJ718
[7] Blei, D. M.; Ng, A. Y.; Jordan, M. I., “Latent Dirichlet Allocation, Journal of Machine Learning Research, 3, 993-1022 (2003) · Zbl 1112.68379
[8] Collins, L. M.; Lanza, S. T., Behavioral, and Health Sciences, Latent Class and Latent Transition Analysis: With Applications in the Social (2009), New York: John Wiley & Sons, New York
[9] Corradin, R., Canale, A., and Nipoti, B. (2020), BNPmix: Bayesian Nonparametric Mixture Models. R package version 0.2.7.
[10] Dellaportas, P.; Papageorgiou, I., “Multivariate Mixtures of Normals With Unknown Number of Components, Statistics and Computing, 16, 57-68 (2006) · doi:10.1007/s11222-006-5338-6
[11] Favaro, S.; Hadjicharalambous, G.; Prünster, I., “On a Class of Distributions on the Simplex, Journal of Statistical Planning and Inference, 141, 2987-3004 (2011) · Zbl 1215.62011 · doi:10.1016/j.jspi.2011.03.015
[12] Fraley, C.; Raftery, A. E., “Bayesian Regularization for Normal Mixture Estimation and Model-Based Clustering, Journal of Classification, 24, 155-181 (2007) · Zbl 1159.62302 · doi:10.1007/s00357-007-0004-5
[13] Fruhwirth-Schnatter, S.; Celeux, G.; Robert, C. P., Handbook of Mixture Analysis (2019), New York: Chapman and Hall/CRC, New York · Zbl 1419.62001
[14] Fúquene, J.; Steel, M.; Rossell, D., “On Choosing Mixture Components Via Non-Local Priors, Journal of the Royal Statistical Society, Series B, 81, 809-837 (2019) · Zbl 1429.62243 · doi:10.1111/rssb.12333
[15] Geyer, C. J.; Møller, J., “Simulation Procedures and Likelihood Inference for Spatial Point Processes,”, Scandinavian Journal of Statistics, 21, 359-373 (1994) · Zbl 0809.62089
[16] Green, P. J., “Reversible Jump Markov Chain Monte Carlo Computation and Bayesian Model Determination, Biometrika, 82, 711-732 (1995) · Zbl 0861.62023 · doi:10.1093/biomet/82.4.711
[17] Green, P. J.; Green, P. J.; Hjort, N. L.; Richardson, S., Highly Structured Stochastic Systems, Trans-Dimensional Markov Chain Monte Carlo, 179-198 (2010), Oxford: Oxford University Press, Oxford
[18] Guha, A.; Ho, N.; Nguyen, X., On Posterior Contraction of Parameters and Interpretability in Bayesian Mixture Modeling, Technical (2019)
[19] Hough, J. B.; Krishnapur, M.; Peres, Y.; Viràg, B., “Determinantal Processes and Independence, Probability Surveys, 3, 206-229 (2006) · Zbl 1189.60101 · doi:10.1214/154957806000000078
[20] Hough, J. B.; Krishnapur, M.; Peres, Y.; Viràg, B., Zeros of Gaussian Analytic Functions and Determinantal Point Processes (2009), Providence, RI: American Mathematical Society, Providence, RI · Zbl 1190.60038
[21] James, L. F.; Lijoi, A.; Prünster, I., “Posterior Analysis for Normalized Random Measures With Independent Increments, Scandinavian Journal of Statistics, 36, 76-97 (2009) · Zbl 1190.62052 · doi:10.1111/j.1467-9469.2008.00609.x
[22] Kendall, W. S.; Møller, J., “Perfect Simulation Using Dominating Processes on Ordered Spaces, With Application to Locally Stable Point Processes, Advances in Applied Probability, 32, 844-865 (2000) · Zbl 1123.60309 · doi:10.1239/aap/1013540247
[23] Kleijn, B. J.; van der Vaart, A. W., “Misspecification in Infinite-Dimensional Bayesian Statistics, The Annals of Statistics, 34, 837-877 (2006) · Zbl 1095.62031 · doi:10.1214/009053606000000029
[24] Lavancier, F.; Møller, J.; Rubak, E., “Determinantal Point Process Models and Statistical Inference, Journal of the Royal Statistical Society, 77, 853-877 (2015) · Zbl 1414.62403 · doi:10.1111/rssb.12096
[25] Li, Y.; Lord-Bessen, J.; Shiyko, M.; Loeb, R., “Bayesian Latent Class Analysis Tutorial, Multivariate Behavioral Research, 53, 430-451 (2018) · doi:10.1080/00273171.2018.1428892
[26] Liang, F.; Jin, I. H.; Song, Q.; Liu, J. S., “An Adaptive Exchange Algorithm for Sampling From Distributions With Intractable normalizing Constants, Journal of the American Statistical Association, 111, 377-393 (2016) · doi:10.1080/01621459.2015.1009072
[27] Lijoi, A.; Prünster, I.; Hjort, N.; Holmes, C.; Müller, P.; Walker, S., Bayesian Nonparametrics, Models beyond the Dirichlet process, 80-136 (2010), Cambridge: Cambridge University Press, Cambridge
[28] Lyne, A.-M.; Girolami, M.; Atchadé, Y.; Strathmann, H.; Simpson, D., “On Russian Roulette Estimates for Bayesian Inference With Doubly-Intractable Likelihoods, Statistical Science, 30, 443-467 (2015) · Zbl 1426.62092 · doi:10.1214/15-STS523
[29] Macchi, O., “The Coincidence Approach to Stochastic Point Processes, Advances in Applied Probability, 7, 83-122 (1975) · Zbl 0366.60081 · doi:10.2307/1425855
[30] Miller, J. W.; Dunson, D. B., “Robust Bayesian Inference Via Coarsening, Journal of the American Statistical Association, 114, 1113-1125 (2019) · Zbl 1428.62287 · doi:10.1080/01621459.2018.1469995
[31] Miller, J. W.; Harrison, M. T., “Mixture Models With a Prior on The Number of Components, Journal of the American Statistical Association, 113, 340-356 (2018) · Zbl 1398.62066 · doi:10.1080/01621459.2016.1255636
[32] Molitor, J.; Papathomas, M.; Jerrett, M.; Richardson, S., “Bayesian Profile Regression With an Application to the National Survey of Children’s Health, Biostatistics, 11, 484-498 (2010) · Zbl 1437.62560 · doi:10.1093/biostatistics/kxq013
[33] Møller, J.; O’Reilly, E., “Couplings for Determinantal Point Processes and Their Reduced Palm Distributions With a View to Quantifying Repulsiveness”, Journal of Applied Probability, 58, 469-483 (2021) · Zbl 1476.60087 · doi:10.1017/jpr.2020.101
[34] Møller, J.; Pettitt, A. N.; Reeves, R.; Berthelsen, K. K., “An Efficient Markov Chain Monte Carlo Method for Distributions With Intractable Normalising Constants, Biometrika, 93, 451-458 (2006) · Zbl 1158.62020 · doi:10.1093/biomet/93.2.451
[35] Møller, J.; Waagepetersen, R. P., Statistical Inference and Simulation for Spatial Point Processes (2004), Boca Raton, FL: Chapman and Hall/CRC, Boca Raton, FL · Zbl 1044.62101
[36] Müller, P.; Mitra, R., “Bayesian Nonparametric Inference - Why and How, Bayesian Analysis, 8, 269-302 (2013) · Zbl 1329.62171 · doi:10.1214/13-BA811
[37] Murray, I.; Ghahramani, Z.; MacKay, D. J. C.; Dechter, Rina; Richardson, Thomas, Proceedings of the Twenty-Second Conference on Uncertainty in Artificial Intelligence (UAI’06), MCMC for Doubly-Intractable Distributions, 359-366 (2006), Arlington, VA: AUAI Press, Arlington, VA
[38] Papaspiliopoulos, O.; Roberts, G. O., “Retrospective Markov Chain Monte Carlo Methods for Dirichlet Process Hierarchical Models, Biometrika, 95, 169-186 (2008) · Zbl 1437.62576 · doi:10.1093/biomet/asm086
[39] Petralia, F.; Rao, V.; Dunson, D. B.; Pereira, F.; Burges, C. J. C.; Bottou, L.; Weinberger, K. Q., Advances in Neural Information Processing Systems, 25, Repulsive mixtures, 1889-1897 (2012), Curran Associates, Inc
[40] Quinlan, J. J.; Quintana, F. A.; Page, G. L., “Parsimonious Hierarchical Modeling Using Repulsive Distributions, Test, 30, 445-461 (2020) · Zbl 1479.62045 · doi:10.1007/s11749-020-00726-y
[41] Regazzini, E.; Lijoi, A.; Prünster, I., “Distributional Results for Means of Normalized Random Measures With Independent Increments, The Annals of Statistics, 31, 560-585 (2003) · Zbl 1068.62034 · doi:10.1214/aos/1051027881
[42] Richardson, S.; Green, P. J., “On Bayesian Analysis of Mixtures With an Unknown Number of Components (with discussion), Journal of the Royal Statistical Society, Series B, 59, 731-792 (1997) · Zbl 0891.62020 · doi:10.1111/1467-9868.00095
[43] Xie, F.; Xu, Y., “Bayesian Repulsive Gaussian Mixture Model, Journal of the American Statistical Association, 115, 187-203 (2019) · Zbl 1437.62242 · doi:10.1080/01621459.2018.1537918
[44] Xu, Y.; Müller, P.; Telesca, D., “Bayesian Inference for Latent Biologic Structure With Determinantal Point Processes (DPP, Biometrics, 72, 955-964 (2016) · Zbl 1390.62320 · doi:10.1111/biom.12482
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.