×

Convergence of de Finetti’s mixing measure in latent structure models for observed exchangeable sequences. (English) Zbl 1539.62084

Summary: Mixtures of product distributions are a powerful device for learning about heterogeneity within data populations. In this class of latent structure models, de Finetti’s mixing measure plays the central role for describing the uncertainty about the latent parameters representing heterogeneity. In this paper, posterior contraction theorems for de Finetti’s mixing measure arising from finite mixtures of product distributions will be established; under the setting the number of exchangeable sequences of observed variables increases while sequence length(s) may be either fixed or varied. The role of both the number of sequences and the sequence lengths will be carefully examined. In order to obtain concrete rates of convergence, a first-order identifiability theory for finite mixture models and a family of sharp inverse bounds for mixtures of product distributions will be developed via a harmonic analysis of such latent structure models. This theory is applicable to broad classes of probability kernels composing the mixture model of product distributions for both continuous and discrete domain \(\mathfrak{X} \). Examples of interest include the case the probability kernel is only weakly identifiable in the sense of [N. Ho and X. Nguyen, Ann. Stat. 44, No. 6, 2726–2755 (2016; Zbl 1359.62076)], the case where the kernel is itself a mixture distribution as in hierarchical models, and the case the kernel may not have a density with respect to a dominating measure on an abstract domain \(\mathfrak{X} \), such as Dirichlet processes.

MSC:

62G05 Nonparametric estimation
62F15 Bayesian inference
62G20 Asymptotic properties of nonparametric inference
62H30 Classification and discrimination; cluster analysis (statistical aspects)

Citations:

Zbl 1359.62076

References:

[1] Aldous, D. J. (1985). Exchangeability and related topics. In École d’été de Probabilités de Saint-Flour, XIII—1983. Lecture Notes in Math. 1117 1-198. Springer, Berlin. · Zbl 0549.00022 · doi:10.1007/BFb0099421
[2] ALIPRANTIS, C. D. and BORDER, K. C. (2006). Infinite Dimensional Analysis: A Hitchhiker’s Guide, 3rd ed. Springer, Berlin. · Zbl 1156.46001
[3] Allman, E. S., Matias, C. and Rhodes, J. A. (2009). Identifiability of parameters in latent structure models with many observed variables. Ann. Statist. 37 3099-3132. · Zbl 1191.62003 · doi:10.1214/09-AOS689
[4] Antoniak, C. E. (1974). Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. Ann. Statist. 2 1152-1174. · Zbl 0335.60034 · doi:10.1214/aos/1176342871
[5] Billingsley, P. (1995). Probability and Measure, 3rd ed. Wiley Series in Probability and Mathematical Statistics. Wiley, New York. · Zbl 0822.60002
[6] Bonhomme, S., Jochmans, K. and Robin, J.-M. (2016). Non-parametric estimation of finite mixtures from repeated measurements. J. R. Stat. Soc. Ser. B. Stat. Methodol. 78 211-229. · Zbl 1411.62079 · doi:10.1111/rssb.12110
[7] Camerlenghi, F., Dunson, D. B., Lijoi, A., Prünster, I. and Rodríguez, A. (2019). Latent nested nonparametric priors (with discussion). Bayesian Anal. 14 1303-1356. · Zbl 1436.62108 · doi:10.1214/19-BA1169
[8] Camerlenghi, F., Lijoi, A., Orbanz, P. and Prünster, I. (2019). Distribution theory for hierarchical processes. Ann. Statist. 47 67-92. · Zbl 1478.60151 · doi:10.1214/17-AOS1678
[9] Chen, J. H. (1995). Optimal rate of convergence for finite mixture models. Ann. Statist. 23 221-233. · Zbl 0821.62023 · doi:10.1214/aos/1176324464
[10] CRUZ-MEDINA, I. R., HETTMANSPERGER, T. P. and THOMAS, H. (2004). Semiparametric mixture models and repeated measures: The multinomial cut point model. J. R. Stat. Soc. Ser. C. Appl. Stat. 53 463-474. · Zbl 1111.62302 · doi:10.1111/j.1467-9876.2004.05203.x
[11] ELMORE, R., HALL, P. and NEEMAN, A. (2005). An application of classical invariant theory to identifiability in nonparametric mixtures. Ann. Inst. Fourier (Grenoble) 55 1-28. · Zbl 1137.62035
[12] ELMORE, R. T., HETTMANSPERGER, T. P. and THOMAS, H. (2004). Estimating component cumulative distribution functions in finite mixture models. Comm. Statist. Theory Methods 33 2075-2086. · Zbl 1215.62036 · doi:10.1081/STA-200026574
[13] ELMORE, R. T. and WANG, S. (2003). Identifiability and estimation in finite mixture models with multinomial coefficients. Technical Report 03-04, Penn State Univ.
[14] FELLER, W. (1968). An Introduction to Probability Theory and Its Applications. Vol. I, 3rd ed. Wiley, New York. · Zbl 0155.23101
[15] Ferguson, T. S. (1973). A Bayesian analysis of some nonparametric problems. Ann. Statist. 1 209-230. · Zbl 0255.62037
[16] Ghosal, S. and van der Vaart, A. (2017). Fundamentals of Nonparametric Bayesian Inference. Cambridge Series in Statistical and Probabilistic Mathematics 44. Cambridge Univ. Press, Cambridge. · Zbl 1376.62004 · doi:10.1017/9781139029834
[17] GUHA, A., HO, N. and NGUYEN, X. (2021). On posterior contraction of parameters and interpretability in Bayesian mixture modeling. Bernoulli 27 2159-2188. · Zbl 1473.62218 · doi:10.3150/20-BEJ1275
[18] Hall, P., Neeman, A., Pakyari, R. and Elmore, R. (2005). Nonparametric inference in multivariate mixtures. Biometrika 92 667-678. · Zbl 1152.62327 · doi:10.1093/biomet/92.3.667
[19] Hall, P. and Zhou, X.-H. (2003). Nonparametric estimation of component distributions in a multivariate mixture. Ann. Statist. 31 201-224. · Zbl 1018.62021 · doi:10.1214/aos/1046294462
[20] Heinrich, P. and Kahn, J. (2018). Strong identifiability and optimal minimax rates for finite mixture estimation. Ann. Statist. 46 2844-2870. · Zbl 1420.62215 · doi:10.1214/17-AOS1641
[21] Hettmansperger, T. P. and Thomas, H. (2000). Almost nonparametric inference for repeated measures in mixture models. J. R. Stat. Soc. Ser. B. Stat. Methodol. 62 811-825. · Zbl 0957.62026 · doi:10.1111/1467-9868.00266
[22] HJORT, N. L., HOLMES, C., MÜLLER, P. and WALKER, S. G. (2010). Bayesian Nonparametrics. Cambridge Series in Statistical and Probabilistic Mathematics 28. Cambridge University Press, Cambridge.
[23] HO, N. and NGUYEN, X. (2016). Convergence rates of parameter estimation for some weakly identifiable finite mixtures. Ann. Statist. 44 2726-2755. · Zbl 1359.62076 · doi:10.1214/16-AOS1444
[24] HO, N. and NGUYEN, X. (2016). On strong identifiability and convergence rates of parameter estimation in finite mixtures. Electron. J. Stat. 10 271-307. · Zbl 1332.62095 · doi:10.1214/16-EJS1105
[25] HO, N. and NGUYEN, X. (2019). Singularity structures and impacts on parameter estimation in finite mixtures of distributions. SIAM J. Math. Data Sci. 1 730-758. · Zbl 1499.62460 · doi:10.1137/18M122947X
[26] Kallenberg, O. (2005). Probabilistic Symmetries and Invariance Principles. Probability and Its Applications (New York). Springer, New York. · Zbl 1084.60003
[27] Lindsay, B. G. (1995). Mixture models: theory, geometry and applications. In NSF-CBMS regional conference series in probability and statistics i-163. JSTOR. · Zbl 1163.62326
[28] NGUYEN, X. (2011). Wasserstein distances for discrete measures and convergence in nonparametric mixture models. Technical Report, Univ. of Michigan MI Dept. of Statistics.
[29] Nguyen, X. (2013). Convergence of latent mixing measures in finite and infinite mixture models. Ann. Statist. 41 370-400. · Zbl 1347.62117 · doi:10.1214/12-AOS1065
[30] NGUYEN, X. (2015). Posterior contraction of the population polytope in finite admixture models. Bernoulli 21 618-646. · Zbl 1368.62288 · doi:10.3150/13-BEJ582
[31] Nguyen, X. (2016). Borrowing strengh in hierarchical Bayes: Posterior concentration of the Dirichlet base measure. Bernoulli 22 1535-1571. · Zbl 1360.62103 · doi:10.3150/15-BEJ703
[32] RITCHIE, A., VANDERMEULEN, R. A. and SCOTT, C. (2020). Consistent estimation of identifiable nonparametric mixture models from grouped observations. arXiv preprint. Available at arXiv:2006.07459.
[33] Rodríguez, A., Dunson, D. B. and Gelfand, A. E. (2008). The nested Dirichlet process. J. Amer. Statist. Assoc. 103 1131-1144. · Zbl 1205.62062 · doi:10.1198/016214508000000553
[34] Rousseau, J. and Mengersen, K. (2011). Asymptotic behaviour of the posterior distribution in overfitted mixture models. J. R. Stat. Soc. Ser. B. Stat. Methodol. 73 689-710. · Zbl 1228.62034 · doi:10.1111/j.1467-9868.2011.00781.x
[35] Teh, Y. W., Jordan, M. I., Beal, M. J. and Blei, D. M. (2006). Hierarchical Dirichlet processes. J. Amer. Statist. Assoc. 101 1566-1581. · Zbl 1171.62349 · doi:10.1198/016214506000000302
[36] TEICHER, H. (1967). Identifiability of mixtures of product measures. Ann. Math. Stat. 38 1300-1302. · Zbl 0153.47904 · doi:10.1214/aoms/1177698805
[37] VANDERMEULEN, R. A. and SCOTT, C. D. (2019). An operator theoretic approach to nonparametric mixture models. Ann. Statist. 47 2704-2733. · Zbl 1431.62274 · doi:10.1214/18-AOS1762
[38] WEI, Y. and NGUYEN, X. (2021). Supplement to “Convergence of de Finetti’s mixing measure in latent structure models for observed exchangeable sequences.” https://doi.org/10.1214/21-AOS2120SUPP
[39] WEI, Y. and NGUYEN, X. (2021). Convergence of de Finetti’s mixing measure in latent structure models for observed exchangeable sequences. arXiv preprint. Available at arXiv:2004.05542v3.
[40] WELLNER, J. and VAN DER VAART, A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer Series in Statistics. Springer Science & Business Media. · Zbl 0862.60002
[41] Wu, Y. and Yang, P. (2020). Optimal estimation of Gaussian mixtures via denoised method of moments. Ann. Statist. 48 1981-2007 · Zbl 1455.62075 · doi:10.1214/19-AOS1873
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.