×

On probabilistic inference by weighted model counting. (English) Zbl 1182.68297

Summary: A recent and effective approach to probabilistic inference calls for reducing the problem to one of weighted model counting (WMC) on a propositional knowledge base. Specifically, the approach calls for encoding the probabilistic model, typically a Bayesian network, as a propositional knowledge base in conjunctive normal form (CNF) with weights associated to each model according to the network parameters. Given this CNF, computing the probability of some evidence becomes a matter of summing the weights of all CNF models consistent with the evidence. A number of variations on this approach have appeared in the literature recently, that vary across three orthogonal dimensions. The first dimension concerns the specific encoding used to convert a Bayesian network into a CNF. The second dimensions relates to whether weighted model counting is performed using a search algorithm on the CNF, or by compiling the CNF into a structure that renders WMC a polytime operation in the size of the compiled structure. The third dimension deals with the specific properties of network parameters (local structure) which are captured in the CNF encoding. In this paper, we discuss recent work in this area across the above three dimensions, and demonstrate empirically its practical importance in significantly expanding the reach of exact probabilistic inference. We restrict our discussion to exact inference and model counting, even though other proposals have been extended for approximate inference and approximate model counting.

MSC:

68T37 Reasoning under uncertainty in the context of artificial intelligence

References:

[1] M. Chavira, A. Darwiche, Compiling Bayesian networks with local structure, in: Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI), 2005, pp. 1306-1312; M. Chavira, A. Darwiche, Compiling Bayesian networks with local structure, in: Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI), 2005, pp. 1306-1312
[2] M. Chavira, D. Allen, A. Darwiche, Exploiting evidence in probabilistic inference, in: Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence (UAI), 2005, pp. 112-119; M. Chavira, D. Allen, A. Darwiche, Exploiting evidence in probabilistic inference, in: Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence (UAI), 2005, pp. 112-119
[3] Chavira, M.; Darwiche, A., Encoding cnfs to empower component analysis, (Proceedings of the 9th International Conference on Theory and Applications of Satisfiability Testing (SAT). Proceedings of the 9th International Conference on Theory and Applications of Satisfiability Testing (SAT), Lecture Notes in Computer Science, vol. 4121 (2006), Springer: Springer Berlin, Heidelberg), 61-74
[4] Jensen, F. V.; Lauritzen, S.; Olesen, K., Bayesian updating in recursive graphical models by local computation, Computational Statistics Quarterly, 4, 269-282 (1990) · Zbl 0715.68076
[5] Lauritzen, S. L.; Spiegelhalter, D. J., Local computations with probabilities on graphical structures and their application to expert systems, Journal of Royal Statistics Society, Series B, 50, 2, 157-224 (1988) · Zbl 0684.68106
[6] Zhang, N. L.; Poole, D., Exploiting causal independence in Bayesian network inference, Journal of Artificial Intelligence Research, 5, 301-328 (1996) · Zbl 0900.68384
[7] R. Dechter, Bucket elimination: A unifying framework for probabilistic inference, in: Proceedings of the 12th Conference on Uncertainty in Artificial Intelligence (UAI), 1996, pp. 211-219; R. Dechter, Bucket elimination: A unifying framework for probabilistic inference, in: Proceedings of the 12th Conference on Uncertainty in Artificial Intelligence (UAI), 1996, pp. 211-219
[8] Darwiche, A., Recursive conditioning, Artificial Intelligence, 126, 1-2, 5-41 (2001) · Zbl 0969.68150
[9] F. Jensen, S.K. Andersen, Approximations in Bayesian belief universes for knowledge based systems, in: Proceedings of the Sixth Conference on Uncertainty in Artificial Intelligence (UAI), Cambridge, MA, 1990, pp. 162-169; F. Jensen, S.K. Andersen, Approximations in Bayesian belief universes for knowledge based systems, in: Proceedings of the Sixth Conference on Uncertainty in Artificial Intelligence (UAI), Cambridge, MA, 1990, pp. 162-169
[10] C. Boutilier, N. Friedman, M. Goldszmidt, D. Koller, Context-specific independence in Bayesian networks, in: Proceedings of the 12th Conference on Uncertainty in Artificial Intelligence (UAI), 1996, pp. 115-123; C. Boutilier, N. Friedman, M. Goldszmidt, D. Koller, Context-specific independence in Bayesian networks, in: Proceedings of the 12th Conference on Uncertainty in Artificial Intelligence (UAI), 1996, pp. 115-123
[11] D. Larkin, R. Dechter, Bayesian inference in the presence of determinism, in: C.M. Bishop, B.J. Frey (Eds.), Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics, Jan 3-6, 2003, Key West, FL; D. Larkin, R. Dechter, Bayesian inference in the presence of determinism, in: C.M. Bishop, B.J. Frey (Eds.), Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics, Jan 3-6, 2003, Key West, FL
[12] Bacchus, F.; Dalmao, S.; Pitassi, T., Value elimination: Bayesian inference via backtracking search, (Proceedings of the 19th Annual Conference on Uncertainty in Artificial Intelligence (UAI-03) (2003), Morgan Kaufmann Publishers: Morgan Kaufmann Publishers San Francisco, CA), 20-28 · Zbl 1182.68294
[13] D. Heckerman, A tractable inference algorithm for diagnosing multiple diseases, in: Proceedings of the Fifth Conference on Uncertainty in Artificial Intelligence, 1989, pp. 174-181; D. Heckerman, A tractable inference algorithm for diagnosing multiple diseases, in: Proceedings of the Fifth Conference on Uncertainty in Artificial Intelligence, 1989, pp. 174-181
[14] Poole, D.; Zhang, N., Exploiting contextual independence in probabilistic inference, Journal of Artificial Intelligence, 18, 263-313 (2003) · Zbl 1056.68144
[15] A. Darwiche, A logical approach to factoring belief networks, in: Proceedings of KR, 2002, pp. 409-420; A. Darwiche, A logical approach to factoring belief networks, in: Proceedings of KR, 2002, pp. 409-420
[16] Sang, T.; Beame, P.; Kautz, H., Solving Bayesian networks by weighted model counting, (Proceedings of the Twentieth National Conference on Artificial Intelligence (AAAI-05), vol. 1 (2005), AAAI Press), 475-482
[17] M. Chavira, A. Darwiche, Compiling Bayesian networks using variable elimination, in: Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI), 2007, pp. 2443-2449; M. Chavira, A. Darwiche, Compiling Bayesian networks using variable elimination, in: Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI), 2007, pp. 2443-2449
[18] T. Sang, F. Bacchus, P. Beame, H.A. Kautz, T. Pitassi, Combining component caching and clause learning for effective model counting, in: SAT, 2004; T. Sang, F. Bacchus, P. Beame, H.A. Kautz, T. Pitassi, Combining component caching and clause learning for effective model counting, in: SAT, 2004
[19] T. Sang, P. Beame, H.A. Kautz, Heuristics for fast exact model counting, in: SAT, 2005, pp. 226-240; T. Sang, P. Beame, H.A. Kautz, Heuristics for fast exact model counting, in: SAT, 2005, pp. 226-240 · Zbl 1128.68481
[20] Darwiche, A., On the tractability of counting theory models and its application to belief revision and truth maintenance, Journal of Applied Non-Classical Logics, 11, 1-2, 11-34 (2001) · Zbl 1033.03505
[21] Pearl, J., Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (1988), Morgan Kaufmann Publishers, Inc.: Morgan Kaufmann Publishers, Inc. San Mateo, CA
[22] Chavira, M.; Darwiche, A.; Jaeger, M., Compiling relational Bayesian networks for exact inference, International Journal of Approximate Reasoning, 42, 1-2, 4-20 (May 2006) · Zbl 1096.68747
[23] Roth, D., On the hardness of approximate reasoning, Artificial Intelligence, 82, 1-2, 273-302 (1996) · Zbl 1506.68143
[24] F. Bacchus, S. Dalmao, T. Pitassi, Algorithms and complexity results for #sat and Bayesian inference, in: FOCS, 2003, pp. 340-351; F. Bacchus, S. Dalmao, T. Pitassi, Algorithms and complexity results for #sat and Bayesian inference, in: FOCS, 2003, pp. 340-351
[25] Davis, M.; Logemann, G.; Loveland, D., A machine program for theorem proving, CACM, 5, 394-397 (1962) · Zbl 0217.54002
[26] A. Darwiche, New advances in compiling CNF to decomposable negational normal form, in: Proceedings of European Conference on Artificial Intelligence, 2004, pp. 328-332; A. Darwiche, New advances in compiling CNF to decomposable negational normal form, in: Proceedings of European Conference on Artificial Intelligence, 2004, pp. 328-332
[27] Dechter, R.; Mateescu, R., AND/OR search spaces for graphical models, Artificial Intelligence, 171, 2-3, 73-106 (2007) · Zbl 1168.68549
[28] W. Wei, B. Selman, A new approach to model counting, in: SAT, 2005, pp. 324-339; W. Wei, B. Selman, A new approach to model counting, in: SAT, 2005, pp. 324-339 · Zbl 1128.68487
[29] W. Wei, J. Erenrich, B. Selman, Towards efficient sampling: Exploiting random walk strategies, in: AAAI, 2004, pp. 670-676; W. Wei, J. Erenrich, B. Selman, Towards efficient sampling: Exploiting random walk strategies, in: AAAI, 2004, pp. 670-676
[30] R. Bayardo, J. Pehoushek, Counting models using connected components, in: AAAI, 2000, pp. 157-162; R. Bayardo, J. Pehoushek, Counting models using connected components, in: AAAI, 2000, pp. 157-162
[31] Darwiche, A., A compiler for deterministic, decomposable negation normal form, (Proceedings of the Eighteenth National Conference on Artificial Intelligence (AAAI) (2002), AAAI Press: AAAI Press Menlo Park, CA), 627-634
[32] F. Bacchus, S. Dalmao, T. Pitassi, Dpll with caching: A new algorithm for #SAT and Bayesian inference, Electronic Colloquium on Computational Complexity (ECCC) 10 (003); F. Bacchus, S. Dalmao, T. Pitassi, Dpll with caching: A new algorithm for #SAT and Bayesian inference, Electronic Colloquium on Computational Complexity (ECCC) 10 (003) · Zbl 1182.68294
[33] Darwiche, A.; Marquis, P., A knowledge compilation map, Journal of Artificial Intelligence Research, 17, 229-264 (2002) · Zbl 1045.68131
[34] J. Huang, A. Darwiche, Dpll with a trace: From sat to knowledge compilation, in: Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI), 2005, pp. 156-162; J. Huang, A. Darwiche, Dpll with a trace: From sat to knowledge compilation, in: Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI), 2005, pp. 156-162
[35] Jaeger, M., Relational Bayesian networks, (Geiger, D.; Shenoy, P. P., Proceedings of the 13th Conference of Uncertainty in Artificial Intelligence (UAI-13) (1997), Morgan Kaufmann, Providence: Morgan Kaufmann, Providence USA), 266-273
[36] Darwiche, A., A differential approach to inference in Bayesian networks, Journal of the ACM, 50, 3, 280-305 (2003) · Zbl 1325.68226
[37] Hayes, J. P., Introduction to Digital Logic Design (1993), Addison Wesley
[38] Mirsalehi, M. M.; Gaylord, T. K., Logical minimization of multilevel coded functions, Applied Optics, 25, 3078-3088 (1986)
[39] Shachter, R. D., Evaluating influence diagrams, Operations Research, 34, 6, 871-882 (1986)
[40] Ross, S., Evidence absorption and propagation through evidence reversals, (Proceedings of the 5th Annual Conference on Uncertainty in Artificial Intelligence (UAI-90) (1990), Elsevier Science Publishing Company, Inc.: Elsevier Science Publishing Company, Inc. New York) · Zbl 0721.68076
[41] Park, J.; Darwiche, A., Approximating MAP using stochastic local search, (Proceedings of the 17th Conference on Uncertainty in Artificial Intelligence (UAI) (2001), Morgan Kaufmann Publishers, Inc.: Morgan Kaufmann Publishers, Inc. San Francisco, CA), 403-410
[42] J. Park, A. Darwiche, Solving map exactly using systematic search, in: Proceedings of the 19th Conference on Uncertainty in Artificial Intelligence (UAI), 2003, pp. 459-468; J. Park, A. Darwiche, Solving map exactly using systematic search, in: Proceedings of the 19th Conference on Uncertainty in Artificial Intelligence (UAI), 2003, pp. 459-468
[43] C. Yuan, T.-C. Lu, M. Druzdzel, Annealed MAP, in: Proceedings of the 20th Annual Conference on Uncertainty in Artificial Intelligence (UAI), 2004, pp. 628-635; C. Yuan, T.-C. Lu, M. Druzdzel, Annealed MAP, in: Proceedings of the 20th Annual Conference on Uncertainty in Artificial Intelligence (UAI), 2004, pp. 628-635
[44] Fishelson, M.; Geiger, D., Exact genetic linkage computations for general pedigrees, Bioinformatics, 18, 1, 189-198 (2002)
[45] M. Fishelson, N. Dovgolevsky, D. Geiger, Maximum likelihood haplotyping for general pedigrees, Tech. Rep. CS-2004-13, Technion, Haifa, Israel, 2004; M. Fishelson, N. Dovgolevsky, D. Geiger, Maximum likelihood haplotyping for general pedigrees, Tech. Rep. CS-2004-13, Technion, Haifa, Israel, 2004
[46] Dempster, A.; Laird, N.; Rubin, D., Maximum likelihood from incomplete data via the EM algorithm, Journal of the Royal Statistical Society, 39, 1, 1-38 (1977) · Zbl 0364.62022
[47] Chan, H.; Darwiche, A., Sensitivity analysis in Bayesian networks: From single to multiple parameters, (Proceedings of the Twentieth Conference on Uncertainty in Artificial Intelligence (UAI) (2004), AUAI Press: AUAI Press Arlington, Virginia), 67-75
[48] Park, J.; Darwiche, A., A differential semantics for jointree algorithms, Artificial Intelligence, 156, 197-216 (2004) · Zbl 1085.68685
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.