×

Robust inference of trees. (English) Zbl 1177.68217

Summary: This paper is concerned with the reliable inference of optimal tree-approximations to the dependency structure of an unknown distribution generating data. The traditional approach to the problem measures the dependency strength between random variables by the index called mutual information. In this paper reliability is achieved by Walley’s imprecise Dirichlet model, which generalizes Bayesian learning with Dirichlet priors. Adopting the imprecise Dirichlet model results in posterior interval expectation for mutual information, and in a set of plausible trees consistent with the data. Reliable inference about the actual tree is achieved by focusing on the substructure common to all the plausible trees. We develop an exact algorithm that infers the substructure in time \(O (m^{4}), m\) being the number of random variables. The new algorithm is applied to a set of data sampled from a known distribution. The method is shown to reliably infer edges of the actual tree even when the data are very scarce, unlike the traditional approach. Finally, we provide lower and upper credibility limits for mutual information under the imprecise Dirichlet model. These enable the previous developments to be extended to a full inferential method for trees.

MSC:

68T37 Reasoning under uncertainty in the context of artificial intelligence
90C35 Programming involving graphs or networks
05C05 Trees
41A58 Series expansions (e.g., Taylor, Lidstone series, but not Fourier series)
62G35 Nonparametric robustness
62H20 Measures of association (correlation, canonical correlation, etc.)

Software:

BayesDA

References:

[1] M. Abramowitz and I.A. Stegun, eds., Handbook of Mathematical Functions (Dover, 1974).
[2] I.D. Aron and P. Van Hentenryck, On the complexity of the robust spanning tree problem with internal data, Operations Research Letters 32 (2004) 36–40. · Zbl 1056.90114 · doi:10.1016/S0167-6377(03)00058-0
[3] J.-M. Bernard, 2001, Non-parametric inference about an unknown mean using the imprecise Dirichlet model, in: ISIPTA’01, eds. G. de Cooman, T. Fine and T. Seidenfeld (The Netherlands, 2001) pp. 40–50.
[4] J.-M. Bernard, An introduction to the imprecise Dirichlet model for multinomial data, International Journal of Approximate 39(2–3) (2005) 123–150. · Zbl 1066.62003 · doi:10.1016/j.ijar.2004.10.002
[5] C.K. Chow and C.N. Liu, Approximating discrete probability distributions with dependence tress, IEEE Transactions on Information Theory, IT-14(3) (1968) 462–468. · Zbl 0165.22305 · doi:10.1109/TIT.1968.1054142
[6] N. Friedman, D. Geiger and M. Goldszmidt, Bayesian networks classifiers, Machine Learning 29(2/3) (1997) 131–163. · Zbl 0892.68077 · doi:10.1023/A:1007465528199
[7] A. Gelman, J.B. Carlin, H.S. Stern and D.B. Rubin, Bayesian Data Analysis (Chapman, 1995). · Zbl 1279.62004
[8] J.B.S. Haldane, The precision of observed values of small frequencies, Biometrika 35 (1948) 297–300. · Zbl 0039.36205
[9] M. Hutter, Distribution of mutual information, in: Proceedings of NIPS*2001, eds. T.G. Dietterich, S. Vecker and Z. Ghahramani (Cambridge, MA, 2001). · Zbl 1007.68149
[10] M. Hutter, Robust estimators under the imprecise dirichlet model, in: Proc. 3rd International Symposium on Imprecise Probalities and Their Application (ISIPTA-2003), Proceedings in Informatics Vol. 18 (Canada, 2003) pp. 274–289.
[11] M. Hutter and M. Zaffalon, Distribution of mutual information from complete and incomplete data, Computational Statics & Data Analysis 48(3) (2005) 633–657. · Zbl 1429.62054 · doi:10.1016/j.csda.2004.03.010
[12] H. Jeffreys, An invariant form for the prior probability in estimation problems, in: Proceedings Royal Society London A, 186 (1946) pp. 453–461. · Zbl 0063.03050 · doi:10.1098/rspa.1946.0056
[13] M.G. Kendall and A. Stuart, The Advanced Theory of Statistics, 2nd edition. (Griffin, London, 1967). · Zbl 0416.62001
[14] G.D. Kleiter, The posterior probability of Bayers nets with strong dependences, Soft Computing 3 (1999) 162–173.
[15] J.B. Kruskal Jr., On the shortest spanning subtree of a graph and the traveling salesman problem, in: Proceedings of the American Mathematical Society 7 (1956) 48–50. · Zbl 0070.18404
[16] S. Kullback, Information Theory and Statistics (Dover, 1968). · Zbl 0274.62036
[17] S. Kullback and R.A. Leiber, On information and sufficiency, Annals of Mathematical Statistics 22 (1951) 79–86. · Zbl 0042.38403 · doi:10.1214/aoms/1177729694
[18] C. Manski, Partial Identification of Probability Distributions (Department of Economics, Northwestern University, USA: Draft book, 2002). · Zbl 1047.62001
[19] R. Montemanni, A Benders decomposition approach for the robust spanning tree problem with interval data, European Journal of Operational Research. Forthcoming. · Zbl 1102.90050
[20] H. Papadimitriou and K. Steiglitz, Combinatorial Optimization: Algorithms and Complexity (Prentice Hall, New York, 1982). · Zbl 0503.90060
[21] J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (Morgan Kaufmann, San Mateo, 1988). · Zbl 0746.68089
[22] W. Perks, Some observations on inverse probability, Journal of the Institute of Actuaries 73 (1947) 285–312. · Zbl 0031.06001
[23] M. Ramoni and P. Sebastiani, Robust learning with missing data, Machine Learning 45(2) (2001) 147–170. · Zbl 1007.68154 · doi:10.1023/A:1010968702992
[24] T. Verma and J. Pearl, Equivalence and synthesis of causal models, in: UAI’90, eds. P.P. Bonissone, M. Henrion, L.N. Kanal and J.F. Lemmer (New York, 1990) pp. 220–227.
[25] P. Walley, Statistical Reasoning with Imprecise Probabilities (Chapman and Hall, New York, 1991). · Zbl 0732.62004
[26] P. Walley, Inferences from multinomial data: learning about a bag of marbles, Journal of the Royal Statistical Society B 58(1) (1996) 3–57. · Zbl 0834.62004
[27] D.H. Wolpert and D.R. Wolf, Estimating functions of distributions from a finite set of samples, Physical Review E 52(6) (1995) 6841–6854. · doi:10.1103/PhysRevE.52.6841
[28] H. Yaman, O.E. Karaşan and M.C. Pinar, The robust spanning tree problem with interval data, Operations Research Letters 29 (2001) 31–40. · Zbl 0981.05029 · doi:10.1016/S0167-6377(01)00078-5
[29] M. Zaffalon, Exact credal treatment of missing data, Journal of Statistical Planning and Inference 105(1) (2002) 105–122. · Zbl 1006.62027 · doi:10.1016/S0378-3758(01)00206-3
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.