Being Bayesian about learning Bayesian networks from ordinal data. (English) Zbl 07885898
Summary: In this paper we propose a Bayesian approach for inferring Bayesian network (BN) structures from ordinal data. Our approach can be seen as the Bayesian counterpart of a recently proposed frequentist approach, referred to as the ‘ordinal structure expectation maximization’ (OSEM) method. Like for the OSEM method, the key idea is to assume that each ordinal variable originates from a Gaussian variable that can only be observed in discretized form, and that the dependencies in the latent Gaussian space can be modeled by BNs; i.e. by directed acyclic graphs (DAGs). Our Bayesian method combines the ‘structure MCMC sampler’ for DAG posterior sampling, a slightly modified version of the ‘Bayesian metric for Gaussian networks having score equivalence’ (BGe score), the concept of the ‘extended rank likelihood’, and a recently proposed algorithm for posterior sampling the parameters of Gaussian BNs. In simulation studies we compare the new Bayesian approach and the OSEM method in terms of the network reconstruction accuracy. The empirical results show that the new Bayesian approach leads to significantly improved network reconstruction accuracies.
MSC:
| 68T37 | Reasoning under uncertainty in the context of artificial intelligence |
Keywords:
Gaussian Bayesian networks; ordinal data; simplified BGe score; extend rank likelihood; Gaussian copula; Markov chain Monte CarloReferences:
| [1] | Pearl, J., Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, 1988, Morgan Kaufmann: Morgan Kaufmann San Francisco, CA, USA |
| [2] | Neapolitan, R., Probabilistic Reasoning in Expert Systems: Theory and Algorithms, 1989, CreateSpace, Scotts: CreateSpace, Scotts Valley CA, USA |
| [3] | Koller, D.; Friedman, N., Probabilistic Graphical Models: Principles and Techniques, Adaptive Computation and Machine Learning series, 2009, MIT Press: MIT Press Cambridge, MA, USA · Zbl 1183.68483 |
| [4] | Chickering, D. M., Learning Bayesian networks is NP-complete, (Fisher, D.; Lenz, H. J., Learning from Data: Artificial Intelligence and Statistics, vol. 5, 1996, Springer: Springer New York), 121-130 |
| [5] | Spirtes, P.; Glymour, C.; Scheines, R., Causation, Prediction, and Search, 2001, MIT Press · Zbl 0981.62001 |
| [6] | Kalisch, M.; Bühlmann, P., Estimating high-dimensional directed acyclic graphs with the PC-algorithm, J. Mach. Learn. Res., 8, 613-636, 2007 · Zbl 1222.68229 |
| [7] | Marella, D.; Vicard, P., Bayesian network structural learning from complex survey data: a resampling based approach, Stat. Methods Appl., 31, 981-1013, 2022 · Zbl 07596129 |
| [8] | Bouckaert, R. R., Properties of Bayesian belief network learning algorithms, (de Mántaras, R. L.; Poole, D., UAI’94: Proceedings of the Tenth Annual Conference on Uncertainty in Artificial Intelligence, 1994, Morgan Kaufmann), 102-109 |
| [9] | Chickering, D.; Geiger, D.; Heckerman, D., Learning Bayesian networks: Search methods and experimental results, (Proceedings of Fifth Conference on Artificial Intelligence and Statistics, Society for Artificial Intelligence in Statistics. Proceedings of Fifth Conference on Artificial Intelligence and Statistics, Society for Artificial Intelligence in Statistics, Ft. Lauderdale, FL, 1995), 112-128 |
| [10] | Cussens, J., Bayesian network learning with cutting planes, (Cozman, F.; Pfeffer, A., UAI 2011: Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence, 2011, AUAI Press), 153-160 |
| [11] | Constantinou, A. C.; Liu, Y.; Kitson, N. K.; Chobtham, K.; Guo, Z., Effective and efficient structure learning with pruning and model averaging strategies, Int. J. Approx. Reason., 151, 292-321, 2022 · Zbl 07629285 |
| [12] | Madigan, D.; York, J., Bayesian graphical models for discrete data, Int. Stat. Rev., 63, 215-232, 1995 · Zbl 0834.62003 |
| [13] | Giudici, P.; Castelo, R., Improving Markov chain Monte Carlo model search for data mining, Mach. Learn., 50, 127-158, 2003 · Zbl 1050.68120 |
| [14] | Friedman, N.; Koller, D., Being Bayesian about network structure: A Bayesian approach to structure discovery in Bayesian networks, Mach. Learn., 50, 95-126, 2003 · Zbl 1033.68104 |
| [15] | Grzegorczyk, M.; Husmeier, D., Improving the structure MCMC sampler for Bayesian networks by introducing a new edge reversal move, Mach. Learn., 71, 265-305, 2008 · Zbl 1470.62075 |
| [16] | Kuipers, J.; Moffa, G., Partition MCMC for inference on acyclic digraphs, J. Am. Stat. Assoc., 112, 282-299, 2017 |
| [17] | Nandy, P.; Hauser, A.; Maathuis, M. H., High-dimensional consistency in score-based and hybrid structure learning, Ann. Stat., 46, 3151-3183, 2018 · Zbl 1411.62144 |
| [18] | Scutari, M.; Graafland, C. E.; Gutiérrez, J. M., Who learns better Bayesian network structures: Constraint-based, score-based or hybrid algorithms?, (International Conference on Probabilistic Graphical Models, PMLR, 2018), 416-427 |
| [19] | Kuipers, J.; Suter, P.; Moffa, G., Efficient sampling and structure learning of Bayesian networks, J. Comput. Graph. Stat., 31, 639-650, 2022 · Zbl 07633197 |
| [20] | Kitson, N. K.; Constantinou, A. C.; Guo, Z.; Liu, Y.; Chobtham, K., A survey of Bayesian network structure learning, Artif. Intell. Rev., 56, 8721-8814, 2023 |
| [21] | Cui, R.; Groot, P.; Heskes, T., Copula PC algorithm for causal discovery from mixed data, (Frasconi, P.; Landwehr, N.; Manco, G.; Vreeken, J., Machine Learning and Knowledge Discovery in Databases, 2016, Springer International Publishing: Springer International Publishing Cham), 377-392 |
| [22] | Tsagris, M.; Borboudakis, G.; Lagani, V.; Tsamardinos, I., Constraint-based causal discovery with mixed data, Int. J. Data Sci. Anal., 6, 19-30, 2018 |
| [23] | Talvitie, T.; Eggeling, R.; Koivisto, M., Learning Bayesian networks with local structure, mixed variables, and exact algorithms, Int. J. Approx. Reason., 115, 69-95, 2019 · Zbl 1468.68173 |
| [24] | Luo, X.; Moffa, G.; Kuipers, J., Learning Bayesian networks from ordinal data, J. Mach. Learn. Res., 22, 1-44, 2021 · Zbl 07626781 |
| [25] | Heckerman, D.; Geiger, D., Learning Bayesian networks: A unification for discrete and Gaussian domains, (Proceedings of the 11th Annual Conference on Uncertainty in Artificial Intelligence (UAI-95), 1995, Morgan Kaufmann: Morgan Kaufmann San Francisco, CA), 274-282 |
| [26] | Geiger, D.; Heckerman, D., Parameter priors for directed acyclic graphical models and the characterization of several probability distributions, Ann. Stat., 30, 1412-1440, 2002 · Zbl 1016.62064 |
| [27] | Kuipers, J.; Moffa, G.; Heckerman, D., Addendum on the scoring of Gaussian directed acyclic graphical models, Ann. Stat., 42, 1689-1691, 2014 · Zbl 1297.62122 |
| [28] | Scutari, M., Learning Bayesian networks with the bnlearn R package, J. Stat. Softw., 35, 1-22, 2010 |
| [29] | Scutari, M., Bayesian network constraint-based structure learning algorithms: Parallel and optimized implementations in the bnlearn R package, J. Stat. Softw., 77, 1-20, 2017 |
| [30] | Scutari, M.; Denis, J.-B., Bayesian Networks: With Examples in R, 2014, Chapman & Hall: Chapman & Hall Boca Raton, FL |
| [31] | Kalisch, M.; Mächler, M.; Colombo, D.; Maathuis, M. H.; Bühlmann, P., Causal inference using graphical models with the R package pcalg, J. Stat. Softw., 47, 1-26, 2012 |
| [32] | Suter, P.; Kuipers, J.; Moffa, G.; Beerenwinkel, N., Bayesian structure learning and sampling of Bayesian networks with the R package BiDAG, J. Stat. Softw., 105, 1-31, 2023 |
| [33] | Hoff, P., Extending the rank likelihood for semiparametric estimation, Ann. Appl. Stat., 1, 265-283, 2007 · Zbl 1129.62050 |
| [34] | Grzegorczyk, M., Being Bayesian about learning Gaussian Bayesian networks from incomplete data, Int. J. Approx. Reason., 160, Article 108954 pp., 2023 · Zbl 07734011 |
| [35] | Dobra, A.; Lenkoski, A.; Rodriguez, A., Bayesian inference for general Gaussian graphical models with application to multivariate lattice data, J. Am. Stat. Assoc., 106, 1418-1433, 2011 · Zbl 1234.62018 |
| [36] | Mohammadi, A.; Abegaz, F.; Heuvel, E.; Wit, E. C., Bayesian Modelling of Dupuytren Disease by Using Gaussian Copula Graphical Models, J. R. Stat. Soc., Ser. C, Appl. Stat., 66, 629-645, 2016 |
| [37] | Lauritzen, S., Graphical Models, 1996, Clarendon Press: Clarendon Press Oxford · Zbl 0907.62001 |
| [38] | Geiger, D.; Heckerman, D., Learning Gaussian networks, (Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence, 1994, Morgan Kaufmann: Morgan Kaufmann San Francisco, CA), 235-243 |
| [39] | Chickering, D. M., Learning Equivalence Classes of Bayesian-Network Structures, J. Mach. Learn. Res., 2, 445-498, 2002 · Zbl 1007.68179 |
| [40] | Shachter, R. D.; Kenley, R., Gaussian influence diagrams, Manag. Sci., 35, 527-550, 1989 |
| [41] | Friedman, N., Learning belief networks in the presence of missing values and hidden variables, (Fisher, D. H., Proceedings of the Fourteenth International Conference on Machine Learning (ICML), 1997, Morgan Kaufmann: Morgan Kaufmann Nashville, Tennessee, USA), 125-133 |
| [42] | Schwarz, G., Estimating the dimension of a model, Ann. Stat., 6, 461-464, 1978 · Zbl 0379.62005 |
| [43] | Wei, G.; Tanner, M., A Monte Carlo implementation of the EM algorithm and the poor man’s data augmentation algorithms, J. Am. Stat. Assoc., 85, 699-704, 1990 |
| [44] | Scutari, M.; Howell, P.; Balding, D. J.; Mackay, I., Multiple Quantitative Trait Analysis Using Bayesian Networks, Genetics, 198, 129-137, 2014 |
| [45] | Sachs, K.; Perez, O.; Pe’er, D.; Lauffenburger, D.; Nolan, G., Protein-signaling networks derived from multiparameter single-cell data, Science, 308, 523-529, 2005 |
| [46] | Hartemink, A. J., Principled Computational Methods for the Validation and Discovery of Genetic Regulatory Networks, 2001, School of Electrical Engineering and Computer Science, Massachusetts Institute of Technology (MIT): School of Electrical Engineering and Computer Science, Massachusetts Institute of Technology (MIT) MA, US, Ph.D. thesis |
| [47] | Mardia, K.; Kent, J.; Bibby, J., Multivariate Analysis, 1979, Academic Press · Zbl 0432.62029 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
