×

A novel method for combining Bayesian networks, theoretical analysis, and its applications. (English) Zbl 1339.68220

Summary: Effective knowledge integration plays a very important role in knowledge engineering and knowledge-based machine learning. The combination of Bayesian networks (BNs) has shown a promising technique in knowledge fusion and the way of combining BNs remains a challenging research topic. An effective method of BNs combination should not impose any particular constraints on the underlying BNs such that the method is applicable to a variety of knowledge engineering scenarios. In general, a sound method of BNs combination should satisfy three fundamental criteria, that is, avoiding cycles, preserving the conditional independencies of BN structures, and preserving the characteristics of individual BN parameters, respectively. However, none of the existing BNs combination method satisfies all the aforementioned criteria. Accordingly, there are only marginal theoretical contributions and limited practical values of previous research on BNs combination. In this paper, following the approach adopted by existing BNs combination methods, we assume that there is an ancestral ordering shared by individual BNs that helps avoid cycles. We first design and develop a novel BNs combination method that focuses on the following two aspects: (1) a generic method for combining BNs that does not impose any particular constraints on the underlying BNs, and (2) an effective approach ensuring that the last two criteria of BNs combination are satisfied. Further through a formal analysis, we compare the properties of the proposed method and that of three classical BNs combination methods, and hence to demonstrate the distinctive advantages of the proposed BNs combination method. Finally, we apply the proposed method in recommender systems for estimating users’ ratings based on their implicit preferences, bank direct marketing for predicting clients’ willingness of deposit subscription, and disease diagnosis for assessing patients’ breast cancer risk.

MSC:

68T05 Learning and adaptive systems in artificial intelligence
68T30 Knowledge representation
68T37 Reasoning under uncertainty in the context of artificial intelligence

Software:

HandTill2001
Full Text: DOI

References:

[1] Pearl, J., Probabilistic Reasoning in Intelligent SystemsNetwork of Plausible Inference (1988), Morgan Kaufmann: Morgan Kaufmann San Francisco, CA
[2] Liao, W.; Ji, Q., Learning Bayesian network parameters under incomplete data with domain knowledge, Pattern Recognit., 42, 3046-3056 (2009) · Zbl 1175.68372
[3] Pernkopf, F.; Wohlmayr, M., Stochastic margin-based structure learning of Bayesian network classifiers, Pattern Recognit., 46, 464-471 (2013) · Zbl 1295.68187
[4] Wong, S. K.M.; Butz, C. J., Constructing the dependency structure of a multiagent probabilistic network, IEEE Trans. Knowl. Data Eng., 13, 395-415 (2001)
[5] Liu, W.-Y.; Yue, K.; Gao, M.-H., Constructing probabilistic graphical model from predicate formulas for fusing logical and probabilistic knowledge, Inf. Sci., 181, 3828-3845 (2011)
[6] Del Sagrado, J.; Moral, S., Qualitative combination of Bayesian networks, Int. J. Intell. Syst., 18, 237-249 (2003) · Zbl 1028.68165
[8] Li, W. H.; Liu, W. Y.; Yue, K., Recovering the global structure from multiple local Bayesian networks, Int. J. Artif. Intell. Tools, 17, 1067-1088 (2008)
[9] Chickering, D. M., Learning equivalence classes of Bayesian network structures, J. Mach. Learn. Res., 2, 445-498 (2002) · Zbl 1007.68179
[10] Schay, G., Introduction to Probability with Statistical Applications (2007), Springer: Springer Boston · Zbl 1135.62001
[13] Liu, W. Y.; Li, W. H.; Yue, K., Intelligent Data Analysis (2007), Science Press of China: Science Press of China Beijing, China
[15] Liu, W. Y.; Song, N., The fuzzy association degree in semantic data model, Fuzzy Set Syst., 117, 203-208 (2001) · Zbl 0985.68024
[19] Avery, C.; Zeckhauser, R., Recommender systems for evaluating computer messages, Commun. ACM, 40, 88-89 (1997)
[21] Fox, S.; Karnawat, K.; Mydland, M.; Dumais, S.; White, T., Evaluating implicit measures to improve web search, ACM Trans. Inf. Syst., 23, 147-168 (2005)
[23] Sajda, P., Machine learning for detection and diagnosis of disease, Annu. Rev. Biomed. Eng., 8, 537-565 (2006)
[26] Bradley, A. P., The use of the area under the ROC curve in the evaluation of machine learning algorithms, Pattern Recognit., 30, 1145-1159 (1997)
[27] Huang, J.; Ling, C. X., Using AUC and accuracy in evaluating learning algorithms, IEEE Trans. Knowl. Data Eng., 17, 299-310 (2005)
[28] Hand, D.; Till, R., A simple generalisation of the area under the ROC curve for multiple class classification problems, Mach. Learn., 45, 171-186 (2001) · Zbl 1007.68180
[29] Dreiseitl, S.; Machado, L. O.; Kittler, H.; Vinterbo, S.; Binder, M., A comparison of machine learning methods for the diagnosis of pigmented skin lesions, J. Biomed. Inf., 34, 28-36 (2001)
[30] Lim, T. S.; Loh, W. Y., A comparison of prediction accuracy, complexity, and training time of thirty-three old and new classification algorithms, Mach. Learn., 40, 203-228 (2000) · Zbl 0969.68669
[31] Meynet, J.; Thiran, J.-P., Information theoretic combination of pattern classifiers, Pattern Recognit., 43, 3412-3421 (2010) · Zbl 1344.68197
[32] Cao, J.; Kwong, S.; Wang, R., A noise-detection based adaboost algorithm for mislabeled data, Pattern Recognit., 45, 4451-4465 (2012) · Zbl 1248.68431
[33] Sohn, S. Y.; Shin, H. W., Experimental study for the comparison of classifier combination methods, Pattern Recognit., 40, 33-40 (2007) · Zbl 1103.68785
[34] Waske, B.; Benediktsson, J. A., Fusion of support vector machines for classification of multisensor data, IEEE Trans. Geosci. Remote Sens., 45, 3858-3866 (2007)
[36] Hansen, L. K.; Salamon, P., Neural network ensembles, IEEE Trans. Pattern Anal. Mach. Intell., 12, 993-1001 (1990)
[37] Zhang, J.; Xu, J.; Liao, S. S., Sampling methods for summarizing unordered vehicle-to-vehicle data streams, Transp. Res. Part CEmerg. Technol., 23, 56-67 (2012)
[38] Zhang, J. D.; Xu, J.; Liao, S. S., Aggregating and sampling methods for processing GPS data streams for traffic state estimation, IEEE Trans. Intell. Transp. Syst., 14, 1629-1641 (2013)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.