Objective Bayesian edge screening and structure selection for Ising networks. (English) Zbl 1486.62307
Summary: The Ising model is one of the most widely analyzed graphical models in network psychometrics. However, popular approaches to parameter estimation and structure selection for the Ising model cannot naturally express uncertainty about the estimated parameters or selected structures. To address this issue, this paper offers an objective Bayesian approach to parameter estimation and structure selection for the Ising model. Our methods build on a continuous spike-and-slab approach. We show that our methods consistently select the correct structure and provide a new objective method to set the spike-and-slab hyperparameters. To circumvent the exploration of the complete structure space, which is too large in practical situations, we propose a novel approach that first screens for promising edges and then only explore the space instantiated by these edges. We apply our proposed methods to estimate the network of depression and alcohol use disorder symptoms from symptom scores of over 26,000 subjects.
Keywords:
Bayesian model selection; Ising model; spike and slab prior; depression; alcohol use disorderSoftware:
IsingFit; BayesReg; qgraph; HDInterval; BGGM; bayesreg; BDgraph; R; WinBUGS; tsbridge; logspline; BayesLogit; ssgraph; JASP; IsingSamplerReferences:
| [1] | American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed). American Psychiatric Association. |
| [2] | Arnold, B. C., & Strauss, D. (1991). Pseudolikelihood estimation: Some examples. Sankhy \(\bar{\text{a}} \): The Indian Journal of Statistics, Series B,53(2), 233-243. · Zbl 0850.62277 |
| [3] | Barber, RF; Drton, M., High dimensional Ising model selection with Bayesian information criteria, Electronic Journal of Statistics, 9, 1, 567-607 (2015) · Zbl 1309.62050 · doi:10.1214/15-EJS1012 |
| [4] | Barbieri, MM; Berger, JO, Optimal predictive model selection, Annals of Statistics, 32, 3, 870-897 (2004) · Zbl 1092.62033 · doi:10.1214/009053604000000238 |
| [5] | Besag, J., Statistical analysis of non-lattice data, Journal of the Royal Statistical Society. Series D (The Statistician), 24, 3, 179-195 (1975) · doi:10.2307/2987782 |
| [6] | Bhattacharyya, A., & Atchade, Y. (2019). Bayesian analysis of high-dimensional discrete graphical models. arXiv. https://arxiv.org/abs/1907.01170 |
| [7] | Borsboom, D.; Cramer, AOJ, Network analysis: An integrative approach to the structure of psychopathology, Annual Review of Clinical Psychology, 9, 91-121 (2013) · doi:10.1146/annurev-clinpsy-050212-185608 |
| [8] | Bühlmann, P.; Kalisch, M.; Meier, L., High-dimensional statistics with a view toward applications in biology, Annual Reviews of Statistics and Its Applications, 1, 255-278 (2014) · doi:10.1146/annurev-statistics-022513-115545 |
| [9] | Carvalho, CM; Scott, JG, Objective Bayesian model selection in Gaussian graphical models, Biometrika, 96, 3, 497-512 (2009) · Zbl 1170.62020 · doi:10.1093/biomet/asp017 |
| [10] | Caspi, A., Houts, R., Belsky, D., Goldman-Mellor, S., Harrington, H., Israel, S., Israel, S., ... Moffit, T. (2014). The p factor: One general psychopathology factor in the structure of psychiatric disorders? Clinical Psychological Science,2(2), 119-137. doi:10.1177/2167702613497473 |
| [11] | Castillo, I., Schmidt-Hieber, J., & van der Vaart, A. (2015). Bayesian linear regression with sparse priors. The Annals of Statistics,43(5), 1986-2018. doi:10.1214/15-AOS1334 · Zbl 1486.62197 |
| [12] | Chen, J., & Chen, Z. (2008). Extended Bayesian information criteria for model selection with large model spaces. Biometrika,95(3), 759-771. doi:10.1093/biomet/asn034 · Zbl 1437.62415 |
| [13] | Consonni, G.; Fouskakis, D.; Liseo, B.; Ntzoufras, I., Prior distributions for objective Bayesian analysis, Bayesian Analysis, 13, 2, 627-679 (2018) · Zbl 1407.62073 · doi:10.1214/18-BA1103 |
| [14] | Constantini, G.; Richetin, J.; Preti, E.; Casini, E.; Epskamp, S.; Perugi, M., Stability and variability of personality networks: A tutorial on recent developments in network psychometrics, Personality and Individual Differences, 136, 68-78 (2019) · doi:10.1016/j.paid.2017.06.011 |
| [15] | Cox, D., The analysis of multivariate binary data, Journal of the Royal Statistical Society. Series B (Applied Statistics), 21, 2, 113-120 (1972) · doi:10.2307/2346482 |
| [16] | Cramer, AOJ; van Borkulo, CD; Giltay, EJ; van der Maas, HLJ; Kendler, KS; Scheffer, M.; Borsboom, D., Major depression as a complex dynamic system, PLoS One, 11, 12, 1-20 (2016) · doi:10.1371/journal.pone.0167490 |
| [17] | Cramer, A. O. J., van der Sluis, S., Noordhof, A., Wichers, M., Geschwind, N., Aggen, S. H., ... Borsboom, D. (2012). Dimensions of normal personality as networks in search of equilibrium: You can’t like parties if you don’t like people. European Journal of Personality,26, 414-431. doi:10.1002/per.1866 |
| [18] | Csiszár, I.; Talata, Z., Consistent estimation of the basic neighborhood of Markov random fields, The Annals of Statistics, 34, 1, 123-145 (2006) · Zbl 1102.62105 · doi:10.1214/009053605000000912 |
| [19] | Dalege, J.; Borsboom, D.; van Harreveld, F.; van den Berg, H.; Conner, M.; van der Maas, HLJ, Towards a formalized acount of attitudes: The causal attitude network (CAN) model, Psychological Review, 123, 1, 2-22 (2016) · doi:10.1037/a0039802 |
| [20] | Dalege, J., Borsboom, D., van Harreveld, F., & van der Maas, H. L. J. (2019). A network perspective on political attitudes: Testing the connectivity hypothesis. Social Psychological and Personality Science,10(6), 746-756. doi:10.1177/1948550618781062 |
| [21] | Dellaportas, P., Forster, J. J., & Ntzoufras, I. (2002). On Bayesian model and variable selection using MCMC. Statistics and Computing,12, 27-36. doi:10.1023/A:1013164120801199 · Zbl 1247.62086 |
| [22] | Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological),39(1), 1-38. · Zbl 0364.62022 |
| [23] | Dobra, A., & Lenkoski, A. (2011). Copula Gaussian graphical models and their application to modeling functional disability data. The Annals of Applied Statistics,5(2A), 969-993. doi:10.1214/10-AOAS397 · Zbl 1232.62046 |
| [24] | Donner, C., & Opper, M. (2017). Inverse Ising problem in continuous time: A latent variable approach. Physical Review E,96(062104), 1-9. doi:10.1103/PhysRevE.96.062104 |
| [25] | Epskamp, S.; Borsboom, D.; Fried, EI, Estimating psychological networks and their accuracy: A tutorial paper, Behavior Research Methods, 50, 195-212 (2018) · doi:10.3758/s13428-017-0862-1 |
| [26] | Epskamp, S.; Cramer, AOJ; Waldorp, LJ; Schmittmann, VD; Borsboom, D., qgraph: Network visualizations of relationships in psychometric data, Journal of Statistical Software, 48, 4, 1-18 (2012) · doi:10.18637/jss.v048.i04 |
| [27] | Epskamp, S., Kruis, J., & Marsman, M. (2017). Estimating psychopathological networks: Be careful what you wish for. PLoS One,12, e0179891. doi:10.1371/journal.pone.0179891 |
| [28] | Epskamp, S., Maris, G., Waldorp, L., & Borsboom, D. (2018). Network psychometrics. In P. Irwing, D. Hughes, & T. Booth (Eds.), Handbook of psychometrics (pp. 953-986). Wiley. |
| [29] | Erdős, P., & Rènyi, A. (1960). On the evolution of random graphs. Publications of the Mathematical Institute of the Hungarian Academy of Sciences,5(1), 17-60. · Zbl 0103.16301 |
| [30] | Geman, S.; Geman, D., Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images, IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 6, 721-741 (1984) · Zbl 0573.62030 · doi:10.1109/TPAMI.1984.4767596 |
| [31] | George, E. I. (1999). Discussion of “Bayesian model averaging and model search strategies by Clyde M. In J. Bernardo, J. Berger, A. Dawid, & A. Smith (Eds.), Bayesian statistics (Vol. 6, pp. 175-177). Oxford University Press. |
| [32] | George, EI; McCulloch, RE, Variable selection via Gibbs sampling, Journal of the American Statistical Association, 88, 423, 881-889 (1993) · doi:10.1080/01621459.1993.10476353 |
| [33] | George, EI; McCulloch, RE, Approaches for Bayesian variable selection, Statistica Sinica, 7, 2, 339-373 (1997) · Zbl 0884.62031 |
| [34] | Geys, H.; Molenberghs, G.; Ryan, LM, Pseudo-likelihood inference for clustered binary data, Communications in Statistics-Theory and Methods, 26, 11, 2743-2767 (2007) · Zbl 0954.62530 · doi:10.1080/03610929708832075 |
| [35] | Gronau, Q. F., Sarafoglou, A., Matzke, D., Boehm, U., Marsman, M., Leslie, D. S., ... Steingroever, H. (2017). A tutorial on bridge sampling. Journal of Mathematical Psychology,81, 80-97. doi:10.1016/j.jmp.2017.09.005 · Zbl 1402.62042 |
| [36] | Huth, K., Luigjes, K., Marsman, M., Goudriaan, A. E., & van Holst, R. J. (in press). Modeling alcohol use disorder as a set of interconnected symptoms—Assessing differences between clinical and population samples and across external factors. Addictive Behaviors. |
| [37] | Ising, E., Beitrag zur theorie des ferromagnetismus, Zeitschrift für Physik, 31, 1, 253-258 (1925) · Zbl 1439.82056 · doi:10.1007/BF02980577 |
| [38] | Jeffreys, H. (1961). Theory of probability (3rd ed.). Oxford University Press. · Zbl 0116.34904 |
| [39] | Kass, RE; Wasserman, L., A reference Bayesian test for nested hypotheses and its relation to the Schwarz criterion, Journal of the American Statistical Association, 90, 431, 928-934 (1995) · Zbl 0851.62020 · doi:10.1080/01621459.1995.10476592 |
| [40] | Kindermann, R., & Snell, J. L. (1980). Markov random fields and their applications (Vol. 1). American Mathematical Society. · Zbl 1229.60003 |
| [41] | Knight, K.; Fu, W., Asymptotics of Lasso-type estimators, The Annals of Statistics, 28, 5, 1356-1378 (2000) · Zbl 1105.62357 · doi:10.1214/aos/1015957397 |
| [42] | Kooperberg, C. (2019). logspline: Routines for logspline density estimation. Retrieved from. https://CRAN.R-project.org/package=logspline R package version 2.1.15 · Zbl 1139.62312 |
| [43] | Kuo, L., & Mallick, B. (1998). Variable selection for regression models. Sankhy \(\bar{\text{ a }} \): The Indian Journal of Statistics, Series B,60(1), 65-81. · Zbl 0972.62016 |
| [44] | Kyung, M., Gill, J., Ghosh, M., & Casella, G. (2010). Penalized regression, standard errors, and Bayesian lassos. Bayesian Analysis, 5(2), 369-412. · Zbl 1330.62289 |
| [45] | Lange, K., A gradient algorithm locally equivalent to the EM algorithm, Journal of the Royal Statistical Society. Series B (Methodological), 57, 2, 425-437 (1995) · Zbl 0813.62021 · doi:10.1111/j.2517-6161.1995.tb02037.x |
| [46] | Lee, M. D., & Wagenmakers, E. (2013). Bayesian cognitive modeling: A practical course. Cambridge University Press. |
| [47] | Love, J., Selker, R., Marsman, M., Jamil, T., Dropmann, D., Verhagen, A. J., ... Wagenmakers, E. (2019). JASP—graphical statistical software for common statistical designs. Journal of Statistical Software,88(2), 1-17. doi:10.18637/jss.v088.i02 |
| [48] | Ly, A., & Wagenmakers, E. J. (2021). Bayes factors for peri-null hypotheses. arXiv. https://arxiv.org/abs/2102.07162. |
| [49] | Makalic, E., & Schmidt, D.F. (2016). High-dimensional Bayesian regularised regression with the BayesReg package. https://arxiv.org/abs/161106649v3 |
| [50] | Marsman, M., Borsboom, D., Kruis, J., Epskamp, S., van Bork, R., Waldorp, L. J., ... Maris, G. K. J. (2018). An introduction to network psychometrics: Relating Ising network models to item response theory models. Multivariate Behavioral Research,53(1), 15-35. doi:10.1080/00273171.2017.1379379 |
| [51] | Marsman, M.; Maris, GKJ; Bechger, TM; Glas, CAW, Bayesian inference for low-rank Ising networks, Scientific Reports, 5, 9050 (2015) · doi:10.1038/srep09050 |
| [52] | Marsman, M., Tanis, C. C., Bechger, T. M., & Waldorp, L. J. (2019). Network psychometrics in educational practice. Maximum likelihood estimation of the Curie-Weiss model. In B. P. Veldkamp & C. Sluijter (Eds.), Theoretical and practical advances in computer-based educational measurement (pp. 93-120). Springer. |
| [53] | Marsman, M.; Wagenmakers, E., Bayesian benefits with JASP, European Journal of Developmental Psychology, 14, 5, 545-555 (2017) · doi:10.1080/17405629.2016.1259614 |
| [54] | Meinshausen, N., & Bühlmann, P. (2006). High-dimensional graphs and variable selection with the Lasso. The Annals of Statistics, 34(3), 1436-1462. · Zbl 1113.62082 |
| [55] | Meng, X.; Wong, WH, Simulating ratios of normalizing constants via a simple identity: A theoretical exploration, Statistica Sinica, 6, 4, 831-860 (1996) · Zbl 0857.62017 |
| [56] | Meredith, M., & Kruschke, J. (2020). HDInterval: Highest (posterior) density intervals. Retrieved from. https://cran.r-project.org/web/packages/HDInterval/index.html |
| [57] | Miller, J. W. (2019). Asymptotic normality, concentration, and coverage of generalized posteriors. arXiv. https://arxiv.org/abs/1907.09611 |
| [58] | Mohammadi, A.; Wit, EC, Bayesian structure learning in sparse Gaussian graphical models, Bayesian Analysis, 10, 1, 109-138 (2015) · Zbl 1335.62056 · doi:10.1214/14-BA889 |
| [59] | Mohammadi, R. (2020). ssgraph: Bayesian graphical estimation using spike-and-slab priors. Retrieved from. https://cran.r-project.org/package=ssgraph |
| [60] | Mohammadi, R., & Wit, E. (2019). BDgraph: An R package for Bayesian structure learning in graphical models. Journal of Statistical Software,89(3) |
| [61] | Narisetty, NN; He, X., Bayesian variable selection with shrinking and diffusing priors, The Annals of Statistics, 42, 2, 789-817 (2014) · Zbl 1302.62158 · doi:10.1214/14-AOS1207 |
| [62] | Ntzoufras, I. (2009). Bayesian modeling using WinBUGS. Wiley. · Zbl 1218.62015 |
| [63] | O’Hara, RB; Sillanpää, MJ, A review of Bayesian variable selection methods: What, how and which, Bayesian Analysis, 4, 1, 85-118 (2009) · Zbl 1330.62291 · doi:10.1214/09-BA403 |
| [64] | Park, T.; Casella, G., The Bayesian lasso, Journal of the American Statistical Association, 103, 482, 681-686 (2008) · Zbl 1330.62292 · doi:10.1198/016214508000000337 |
| [65] | Pensar, J.; Nyman, H.; Niiranen, J.; Corander, J., Marginal pseudo-likelihood learning of discrete Markov network structures, Bayesian Analysis, 12, 4, 1195-1215 (2017) · Zbl 1384.62178 · doi:10.1214/16-BA1032 |
| [66] | Polson, NG; Scott, JG; Windle, J., Bayesian inference for logistic models using Pólya-Gamma latent variables, Journal of the American Statistical Association, 108, 504, 1339-1349 (2013) · Zbl 1283.62055 · doi:10.1080/01621459.2013.829001 |
| [67] | Polson, N. G. , Scott, J. G. , & Windle, J. (2013b). BayesLogit: PolyaGamma sampling. Retrieved from. https://CRAN.R-project.org/package=BayesLogit |
| [68] | Pötscher, BM; Leeb, H., On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding, Journal of Multivariate Analysis, 100, 9, 2065-2082 (2009) · Zbl 1170.62046 · doi:10.1016/j.jmva.2009.06.010 |
| [69] | R Core Team. (2019). R: A language and environment for statistical computing. Vienna, Austria. https://www.R-project.org/ |
| [70] | Raftery, AE, Bayes factors and BIC. Comment on “A critique of the Bayesian information criterion for model selection”, Sociological Methods & Research, 27, 3, 411-427 (1999) · doi:10.1177/0049124199027003005 |
| [71] | Ravikumar, P.; Wainwright, MJ; Lafferty, JD, High-dimensional Ising model selection using \(l_1\)-regularized logistic regression, Annals of Statistics, 38, 3, 1287-1319 (2010) · Zbl 1189.62115 · doi:10.1214/09-AOS691 |
| [72] | Ročková, V., Bayesian estimation of sparse signals with a continuous spike-and-slab prior, The Annals of Statistics, 46, 1, 401-437 (2018) · Zbl 1395.62230 · doi:10.1214/17-AOS1554 |
| [73] | Ročková, V.; George, EI, The EM approach to Bayesian variable selection, Journal of the American Statistical Association, 109, 506, 828-846 (2014) · Zbl 1367.62049 · doi:10.1080/01621459.2013.869223 |
| [74] | Ročková, V.; George, EI, The spike-and-slab lasso, Journal of the American Statistical Association, 113, 521, 431-444 (2018) · Zbl 1398.62186 · doi:10.1080/01621459.2016.1260469 |
| [75] | Savi, AO; Marsman, M.; van der Maas, HLJ; Maris, GKJ, The wiring of intelligence, Perspectives on Psychological Science, 16, 6, 1034-1061 (2019) · doi:10.1177/1745691619866447 |
| [76] | Schwarz, G., Estimating the dimension of a model, The Annals of Statistics, 6, 2, 461-464 (1978) · Zbl 0379.62005 · doi:10.1214/aos/1176344136 |
| [77] | Scott, JG; Berger, JO, Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem, Annals of Statistics, 38, 5, 2587-2619 (2010) · Zbl 1200.62020 · doi:10.1214/10-AOS792 |
| [78] | Storey, J., The positive false discovery rate: A Bayesian interpretation and the q-value, The Annals of Statistics, 31, 6, 2013-2035 (2003) · Zbl 1042.62026 · doi:10.1214/aos/1074290335 |
| [79] | Talhouk, A.; Doucet, A.; Murphy, K., Efficient Bayesian inference for multivariate probit models with sparse inverse covariance matrices, Journal of Computational and Graphical Statistics, 21, 3, 739-757 (2012) · doi:10.1080/10618600.2012.679239 |
| [80] | Tanner, M. (1996). Tools for statistical inference. Methods for the exploration of posterior distributions and likelihood functions. Springer. doi:10.1007/978-1-4612-4024-2 · Zbl 0846.62001 |
| [81] | Tibshirani, R., Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society. Series B (Methodological), 58, 1, 267-288 (1996) · Zbl 0850.62538 · doi:10.1111/j.2517-6161.1996.tb02080.x |
| [82] | Tierney, L.; Kass, RE; Kadane, JB, Fully exponential Laplace approximations to expectations and variances of nonpositive functions, Journal of the American Statistical Association, 84, 407, 710-716 (1989) · Zbl 0682.62012 · doi:10.2307/2289652drton |
| [83] | United States Department of Health and Human Services. (2016). National survey on drug use and health, 2014. Inter-university Consortium for Political and Social Research [distributor]. doi:10.3886/ICPSR36361.v1 |
| [84] | van Borkulo, CD; Borsboom, D.; Epskamp, S.; Blanken, TF; Boschloo, L.; Schoevers, RA; Waldorp, LJ, A new method for constructing networks from binary data, Scientific Reports, 4, 5918. (2014) · doi:10.1038/srep05918 |
| [85] | van Borkulo, C. D., Epskamp, S., & Robitzsch, A. (2016). IsingFit: Fitting Ising models using the eLasso method. Retrieved from. https://CRAN.R-project.org/package=IsingFit (R package version 0.3.1) |
| [86] | van de Geer, S.; Bülmann, P.; Ritov, Y.; Dezeure, R., On asymptotically optimal confidence regions and tests for high-dimensional models, The Annals of Statistics, 42, 3, 1166-1202 (2014) · Zbl 1305.62259 · doi:10.1214/14-AOS1221 |
| [87] | van Erp, S.; Oberski, DL; Mulder, J., Shrinkage priors for Bayesian penalized regression, Journal of Mathematical Psychology, 28, 31-50 (2019) · Zbl 1431.62564 · doi:10.1016/j.jmp.2018.12.004 |
| [88] | van der Maas, HLJ; Kan, KJ; Marsman, M.; Stevenson, CE, Network models for cognitive development and intelligence, Journal of Intelligence, 5, 2, 1-17 (2017) · doi:10.3390/jintelligence5020016 |
| [89] | Wagenmakers, E., A practical solution to the pervasive problems of p values, Psychonomic Bulletin & Review, 14, 779-804 (2007) · doi:10.3758/BF03194105 |
| [90] | Wagenmakers, E., Love, J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., ... Morey, R. D. (2018). Bayesian inference for psychology. Part ii: Example applications with JASP. Psychonomic Bulletin & Review,25(1), 58-76. doi:10.3758/s13423-017-1323-7 |
| [91] | Wagenmakers, E., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Love, J., ... Morey, R. D. (2018). Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications. Psychonomic Bulleting & Review,25(1), 58-76. doi:10.3758/s13423-017-1343-3 |
| [92] | Wang, H., Scaling it up: Stochastic search structure learning in graphical models, Bayesian Analysis, 10, 2, 351-377 (2015) · Zbl 1335.62068 · doi:10.1214/14-BA916 |
| [93] | Williams, D. R. (2021). The confidence interval that wasn’t: Bootstrapped “confidence intervals” in \(L_1\)-regularized partial correlation networks. (PsyArXiv.) doi:10.31234/osf.io/kjh2f |
| [94] | Williams, DR; Mulder, J., Bayesian hypothesis testing for Gaussian graphical models: Conditional independence and order constraints, Journal of Mathematical Psychology, 99, 102441 (2020) · Zbl 1455.91207 · doi:10.1016/j.jmp.2020.102441 |
| [95] | Williams, DR; Mulder, J., BGGM: Bayesian Gaussian graphical models in R, Journal of Open Source Software, 5, 51, 2111 (2020) · doi:10.21105/joss.02111 |
| [96] | Williams, DR; Rast, P.; Pericchi, LR; Mulder, J., Comparing Gaussian graphical models with the posterior predictive distribution and Bayesian model selection, Psychological Methods, 25, 5, 653-672 (2020) · doi:10.1037/met0000254 |
| [97] | Windle, J., Polson, N. G., & Scott, J. G. (2014). Sampling Pólya-Gamma random variates: Alternate and approximate techniques. https://arXiv.org/abs/1405.0506 |
| [98] | Womack, A. J., Fuentes, C., & Taylor-Rodriguez, D. (2015). Model space priors for objective sparse Bayesian regression. https://arXiv.org/abs/1511.04745 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
