×

Enhancing SMT-based weighted model integration by structure awareness. (English) Zbl 07840012

Summary: The development of efficient exact and approximate algorithms for probabilistic inference is a long-standing goal of artificial intelligence research. Whereas substantial progress has been made in dealing with purely discrete or purely continuous domains, adapting the developed solutions to tackle hybrid domains, characterized by discrete and continuous variables and their relationships, is highly non-trivial. Weighted Model Integration (WMI) recently emerged as a unifying formalism for probabilistic inference in hybrid domains. Despite a considerable amount of recent work, allowing WMI algorithms to scale with the complexity of the hybrid problem is still a challenge. In this paper we highlight some substantial limitations of existing state-of-the-art solutions, and develop an algorithm that combines SMT-based enumeration, an efficient technique in formal verification, with an effective encoding of the problem structure. This allows our algorithm to avoid generating redundant models, resulting in drastic computational savings. Additionally, we show how SMT-based approaches can seamlessly deal with different integration techniques, both exact and approximate, significantly expanding the set of problems that can be tackled by WMI technology. An extensive experimental evaluation on both synthetic and real-world datasets confirms the substantial advantage of the proposed solution over existing alternatives. The application potential of this technology is further showcased on a prototypical task aimed at verifying the fairness of probabilistic programs.

MSC:

68T20 Problem solving in the context of artificial intelligence (heuristics, search strategies, etc.)
68T37 Reasoning under uncertainty in the context of artificial intelligence
68Q60 Specification and verification (program logics, model checking, etc.)

Software:

UCI-ml; LattE; MathSAT5

References:

[1] Spallitta, G.; Masina, G.; Morettin, P.; Passerini, A.; Sebastiani, R., SMT-based weighted model integration with structure awareness, (Proceedings of the 38th Conference on Uncertainty in Artificial Intelligence, vol. 180 (2022)), 1876-1885
[2] Hensher, D. A.; Button, K. J., Handbook of Transport Modelling (2007), Emerald Group Publishing Limited
[3] Thrun, S.; Burgard, W.; Fox, D., Probabilistic Robotics, Intelligent Robotics and Autonomous Agents (2005), MIT Press · Zbl 1081.68703
[4] Lee, E. A., Cyber physical systems: design challenges, (The 11th IEEE International Symposium on Object and Component-Oriented Real-Time Distributed Computing (2008)), 363-369
[5] Friedman, N.; Goldszmidt, M., Discretizing continuous attributes while learning Bayesian networks, (ICML (1996))
[6] Kozlov, A.; Koller, D., Nonuniform dynamic discretization in hybrid networks, (UAI (1997))
[7] Lauritzen, S., Propagation of probabilities, means, and variances in mixed graphical association models, J. Am. Stat. Assoc., 87 (1992) · Zbl 0850.62094
[8] Yang, E.; Baker, Y.; Ravikumar, P.; Allen, G.; Liu, Z., Mixed graphical models via exponential families, (AISTATS (2014))
[9] Molina, A.; Vergari, A.; Di Mauro, N.; Natarajan, S.; Esposito, F.; Kersting, K., Mixed sum-product networks: a deep architecture for hybrid domains, (AAAI (2018))
[10] Bueff, A.; Speichert, S.; Belle, V., Tractable querying and learning in hybrid domains via sum-product networks, (KR Workshop on Hybrid Reasoning and Learning (2018))
[11] Belle, V.; Passerini, A.; Van Den Broeck, G., Probabilistic inference in hybrid domains by weighted model integration, (Proceedings of the 24th International Joint Conference on Artificial Intelligence (2015)), 2770-2776
[12] Morettin, P.; Passerini, A.; Sebastiani, R., Efficient weighted model integration via SMT-based predicate abstraction, (Proceedings of the 26th International Joint Conference on Artificial Intelligence (2017)), 720-728
[13] Morettin, P.; Zuidberg Dos Martires, P.; Kolb, S.; Passerini, A., Hybrid probabilistic inference with logical and algebraic constraints: a survey, (Proceedings of the 30th International Joint Conference on Artificial Intelligence (2021)), 4533-4542
[14] Chavira, M.; Darwiche, A., On probabilistic inference by weighted model counting, Artif. Intell., 172, 772-799 (2008) · Zbl 1182.68297
[15] Barrett, C. W.; Sebastiani, R.; Seshia, S. A.; Tinelli, C., Satisfiability modulo theories, (Handbook of Satisfiability, vol. 336 (2021)), 1267-1329
[16] Sang, T.; Bacchus, F.; Beame, P.; Kautz, H. A.; Pitassi, T., Combining component caching and clause learning for effective model counting, (International Conference on Theory and Applications of Satisfiability Testing (2004)), 20-28
[17] Bacchus, F.; Dalmao, S.; Pitassi, T., Solving #SAT and Bayesian inference with backtracking search, J. Artif. Intell. Res., 34, 391-442 (2009) · Zbl 1182.68294
[18] Belle, V.; den Broeck, G. V.; Passerini, A., Component caching in hybrid domains with piecewise polynomial densities, Proc. AAAI Conf. Artif. Intell., 30 (2016)
[19] Zeng, Z.; den Broeck, G. V., Efficient search-based weighted model integration, (Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence (2020)), 175-185
[20] Zeng, Z.; Morettin, P.; Yan, F.; Vergari, A.; Broeck, G. V.D., Scaling up hybrid probabilistic inference with logical and arithmetic constraints via message passing, (Proceedings of the 37th International Conference on Machine Learning (2020)), 10990-11000
[21] Zeng, Z.; Morettin, P.; Yan, F.; Vergari, A.; Van den Broeck, G., Probabilistic inference with algebraic constraints: theoretical limits and practical approximations, Adv. Neural Inf. Process. Syst., 33, 11564-11575 (2020)
[22] Abboud, R.; Ceylan, İ.İ.; Dimitrov, R., On the approximability of weighted model integration on DNF structures, (Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning, vol. 17 (2020)), 828-837
[23] Belle, V.; Van den Broeck, G.; Passerini, A.; Meila, M.; Heskes, T., Hashing-based approximate probabilistic inference in hybrid domains, (Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence (2015)), 141-150
[24] Darwiche, A.; Marquis, P., A knowledge compilation map, J. Artif. Intell. Res., 17, 229-264 (2002) · Zbl 1045.68131
[25] Morettin, P.; Passerini, A.; Sebastiani, R., Advanced SMT techniques for weighted model integration, Artif. Intell., 275, 1-27 (2019) · Zbl 1478.68328
[26] Lahiri, S. K.; Nieuwenhuis, R.; Oliveras, A., SMT techniques for fast predicate abstraction, (Computer Aided Verification (2006)), 424-437
[27] Sanner, S.; Abbasnejad, E., Symbolic variable elimination for discrete and continuous graphical models, (26th AAAI Conference on Artificial Intelligence (2012)), 1954-1960
[28] Kolb, S.; Mladenov, M.; Sanner, S.; Belle, V.; Kersting, K., Efficient symbolic integration for probabilistic inference, (Proceedings of the 27th International Joint Conference on Artificial Intelligence (2018)), 5031-5037
[29] Dos Martires, P. Z.; Dries, A.; Raedt, L. D., Exact and approximate weighted model integration with probability density functions using knowledge compilation, Proc. AAAI Conf. Artif. Intell., 33, 7825-7833 (2019)
[30] Kolb, S.; Martires, P. Z.D.; Raedt, L. D., How to exploit structure while solving weighted model integration problems, (Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence (2020)), 744-754
[31] Feldstein, J.; Belle, V., Lifted reasoning meets weighted model integration, (Proceedings of the 37th Conference on Uncertainty in Artificial Intelligence (2021)), 322-332
[32] Lauritzen, S. L.; Jensen, F., Stable local computation with conditional Gaussian distributions, Stat. Comput., 11, 191-203 (2001)
[33] Yang, E.; Baker, Y.; Ravikumar, P.; Allen, G.; Liu, Z., Mixed graphical models via exponential families, (Proceedings of the 17th International Conference on Artificial Intelligence and Statistics (2014)), 1042-1050
[34] Molina, A.; Vergari, A.; Di Mauro, N.; Natarajan, S.; Esposito, F.; Kersting, K., Mixed sum-product networks: a deep architecture for hybrid domains, Proc. AAAI Conf. Artif. Intell., 32, 3828-3835 (2018)
[35] Afshar, H. M.; Sanner, S.; Webers, C., Closed-form Gibbs sampling for graphical models with algebraic constraints, (30th AAAI Conference on Artificial Intelligence (2016)), 3287-3293
[36] Gogate, V.; Dechter, R., Approximate inference algorithms for hybrid Bayesian networks with discrete constraints, (Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence (2005)), 209-216
[37] Sanner, S.; Delgado, K. V.; de Barros, L. N., Symbolic dynamic programming for discrete and continuous state MDPs, (Proceedings of the 17th Conference on Uncertainty in Artificial Intelligence (2011)), 643-652
[38] De Salvo Braz, R.; O’Reilly, C.; Gogate, V.; Dechter, R., Probabilistic inference modulo theories, (Proceedings of the 25th International Joint Conference on Artificial Intelligence (2016)), 3591-3599
[39] Fredrikson, M.; Jha, S., Satisfiability modulo counting: a new approach for analyzing privacy properties, (Proceedings of the Joint Meeting of the Twenty-Third EACSL Annual Conference on Computer Science Logic (CSL) and the Twenty-Ninth Annual ACM/IEEE Symposium on Logic in Computer Science (LICS) (2014)), 1-10
[40] Chistikov, D.; Dimitrova, R.; Majumdar, R., Approximate counting in SMT and value estimation for probabilistic programs, (International Conference on Tools and Algorithms for the Construction and Analysis of Systems (2015), Springer), 320-334 · Zbl 1420.68194
[41] Albarghouthi, A.; D’Antoni, L.; Drews, S.; Nori FairSquare, A. V., Probabilistic verification of program fairness, Proc. ACM Program. Lang., 1, 1-30 (2017)
[42] Sebastiani, R., Are you satisfied by this partial assignment? (2020), CoRR
[43] Möhle, S.; Sebastiani, R.; Biere, A., Four flavors of entailment, (Theory and Applications of Satisfiability Testing, vol. 12178 (2020)), 62-71 · Zbl 07331012
[44] Cimatti, A.; Griggio, A.; Schaafsma, B. J.; Sebastiani, R., The MathSAT5 SMT solver, (Tools and Algorithms for the Construction and Analysis of Systems (2013)), 93-107 · Zbl 1381.68153
[45] Sang, T.; Bearne, P.; Kautz, H., Performing Bayesian inference by weighted model counting, (Proceedings of the 20th National Conference on Artificial Intelligence (2005)), 475-481
[46] Tseitin, G. S., On the complexity of derivation in propositional calculus, (Automation of Reasoning (1983)), 466-483
[47] Plaisted, D. A.; Greenbaum, S., A structure-preserving clause form translation, J. Symb. Comput., 2, 293-304 (1986) · Zbl 0636.68119
[48] Masina, G.; Spallitta, G.; Sebastiani, R., On CNF conversion for disjoint SAT enumeration, (26th International Conference on Theory and Applications of Satisfiability Testing. 26th International Conference on Theory and Applications of Satisfiability Testing, SAT 2023. 26th International Conference on Theory and Applications of Satisfiability Testing. 26th International Conference on Theory and Applications of Satisfiability Testing, SAT 2023, LIPIcs, vol. 271 (2023), Schloss Dagstuhl - Leibniz-Zentrum für Informatik), 15:1-15:16
[49] De Loera, J. A.; Hemmecke, R.; Tauzer, J.; Yoshida, R., Effective lattice point counting in rational convex polytopes, J. Symb. Comput., 38, 1273-1302 (2004) · Zbl 1137.52303
[50] Gehr, T.; Misailovic, S.; Vechev, M., PSI: exact symbolic inference for probabilistic programs, (Computer Aided Verification. Computer Aided Verification, Lecture Notes in Computer Science (2016)), 62-83
[51] Ram, P.; Gray, A. G., Density estimation trees, (Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2011)), 627-635
[52] Dua, D.; Graff, C., UCI machine learning repository (2017)
[53] Morettin, P.; Kolb, S.; Teso, S.; Passerini, A., Learning weighted model integration distributions, Proc. AAAI Conf. Artif. Intell., 34, 5224-5231 (2020)
[54] Newman, M. E.J.; Barkema, G. T., Monte Carlo Methods in Statistical Physics (1999), Clarendon Press · Zbl 1012.82019
[55] Chalkis, A.; Fisikopoulos Volesti, V., Volume approximation and sampling for convex polytopes in R, R J., 13, 561 (2021)
[56] Chalkis, A.; Emiris, I. Z.; Fisikopoulos, V., Practical volume estimation by a new annealing schedule for cooling convex bodies (2019), CoRR
[57] Chalkis, A.; Fisikopoulos, V.; Tsigaridas, E.; Zafeiropoulos, H., Geometric algorithms for sampling the flux space of metabolic networks (2021), arXiv
[58] Gelman, A.; Rubin, D. B., Inference from iterative simulation using multiple sequences, Stat. Sci., 7, 457-472 (1992) · Zbl 1386.65060
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.