A maximum entropy approach to learn Bayesian networks from incomplete data. (English) Zbl 1364.62061
Polpo, Adriano (ed.) et al., Interdisciplinary Bayesian statistics. EBEB 2014. Proceedings of the 12th Brazilian meeting on Bayesian statistics, Atibaia, Brazil, March 10–14, 2014. Cham: Springer (ISBN 978-3-319-12453-7/hbk; 978-3-319-12454-4/ebook). Springer Proceedings in Mathematics & Statistics 118, 69-82 (2015).
Summary: This chapter addresses the problem of estimating the parameters of a Bayesian network from incomplete data. This is a hard problem, which for computational reasons cannot be effectively tackled by a full Bayesian approach. The work around is to search for the estimate with maximum posterior probability. This is usually done by selecting the highest posterior probability estimate among those found by multiple runs of Expectation-Maximization with distinct starting points. However, many local maxima characterize the posterior probability function, and several of them have similar high probability. We argue that high probability is necessary but not sufficient in order to obtain good estimates. We present an approach based on maximum entropy to address this problem and describe a simple and effective way to implement it. Experiments show that our approach produces significantly better estimates than the most commonly used method.
For the entire collection see [Zbl 1310.62007].
For the entire collection see [Zbl 1310.62007].
MSC:
62F15 | Bayesian inference |
68T35 | Theory of languages and software systems (knowledge-based systems, expert systems, etc.) for artificial intelligence |
94A17 | Measures of information, entropy |