Naïve possibilistic network classifiers. (English) Zbl 1192.68519
Summary: Naïve Bayesian network classifiers have proved their effectiveness to accomplish the classification task, even if they work under the strong assumption of independence of attributes in the context of the class node. However, as all of them are based on probability theory, they run into problems when they are faced with imperfection. This paper proposes a new approach of classification under the possibilistic framework with naïve classifiers. To output the naïve possibilistic network classifier, two procedures are studied namely the building phase, which deals with imperfect (imprecise/uncertain) dataset attributes and classes, and the classification phase, which is used to classify new instances that may be characterized by imperfect attributes. To improve the performance of our classifier, we propose two extensions namely selective naïve possibilistic classifier and semi-naïve possibilistic classifier. Experimental study has shown naïve Bayes style possibilistic classifier, and is efficient in the imperfect case.
MSC:
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 68T35 | Theory of languages and software systems (knowledge-based systems, expert systems, etc.) for artificial intelligence |
Keywords:
possibility theory; classification; naïve Bayes classifier; possibilistic classifier; aggregation operatorsSoftware:
UCI-mlReferences:
| [1] | N. Ben Amor, S. Benferhat, D. Dubois, K. Mellouli, H. Prade, A theoretical framework for possibilistic independence in a weakly ordered setting, International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems (2002) 117-155.; N. Ben Amor, S. Benferhat, D. Dubois, K. Mellouli, H. Prade, A theoretical framework for possibilistic independence in a weakly ordered setting, International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems (2002) 117-155. · Zbl 1084.68126 |
| [2] | N. Ben Amor, S. Benferhat, D. Dubois, K. Mellouli, H. Prade, Anytime propagation algorithm for min-based possibilistic graphs, Soft Computing, A fusion of Foundations, Methodologies and Applications, Vol. 8, Springer, Berlin, 2003, pp. 150-161.; N. Ben Amor, S. Benferhat, D. Dubois, K. Mellouli, H. Prade, Anytime propagation algorithm for min-based possibilistic graphs, Soft Computing, A fusion of Foundations, Methodologies and Applications, Vol. 8, Springer, Berlin, 2003, pp. 150-161. |
| [3] | N. Ben Amor, S. Benferhat, Z. Elouedi, Qualitative classification and evaluation, in: Possibilistic Decision Trees, FUZZ-IEEE 2, 2004, pp. 653-657.; N. Ben Amor, S. Benferhat, Z. Elouedi, Qualitative classification and evaluation, in: Possibilistic Decision Trees, FUZZ-IEEE 2, 2004, pp. 653-657. |
| [4] | C.M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, New York, 1996.; C.M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, New York, 1996. · Zbl 0868.68096 |
| [5] | C. Borgelt, R. Kruse, Efficient maximum projection of database-induced multivariate possibility distributions, in: Proc. 7th IEEE Internat. Conf. on Fuzzy Systems, USA, 1998, pp. 663-668.; C. Borgelt, R. Kruse, Efficient maximum projection of database-induced multivariate possibility distributions, in: Proc. 7th IEEE Internat. Conf. on Fuzzy Systems, USA, 1998, pp. 663-668. |
| [6] | C. Borgelt, J. Gebhardt, A naïve Bayes style possibilistic classifier, in: Proc. 7th European Congress on Intelligent Techniques and Soft Computing, Germany, 1999, pp. 556-565.; C. Borgelt, J. Gebhardt, A naïve Bayes style possibilistic classifier, in: Proc. 7th European Congress on Intelligent Techniques and Soft Computing, Germany, 1999, pp. 556-565. |
| [7] | Cover, T. M.; Hart, P. E., Nearest neighbour pattern classification, IEEE Transactions on Information Theory, 13, 21-27 (1967) · Zbl 0154.44505 |
| [8] | Domingos, P.; Pazzani, M., On the optimality of the simple Bayesian classifier under zero-one loss, Machine Learning, 29, 103-130 (1997) · Zbl 0892.68076 |
| [9] | Dubois, D.; Prade, H., Possibility Theory (1988), Plenum Press: Plenum Press New York · Zbl 0645.68108 |
| [10] | Dubois, D.; Prade, H., Possibility Theory: An Approach to computerized Processing of Uncertainty (1994), Plenum Press: Plenum Press New York |
| [11] | Dubois, D.; Prade, H., Possibility theory: qualitative and quantitative aspects, (Gabbay, D.; Smets, P., Handbook of Defeasible Reasoning and Uncertainty Management Systems, Vol. 1 (1998), Kluwer Academic Publishers: Kluwer Academic Publishers Dordrecht), 169-226 · Zbl 0924.68182 |
| [12] | D. Dubois, H. Prade, Possibility theory in information fusion, in: Proc. 3rd Internat. Conf. on Information Fusion, Paris, 2000, pp. 21-34.; D. Dubois, H. Prade, Possibility theory in information fusion, in: Proc. 3rd Internat. Conf. on Information Fusion, Paris, 2000, pp. 21-34. · Zbl 1001.68179 |
| [13] | Duda, R.; Hart, P., Pattern Classification and Scene Analysis (1973), Wiley: Wiley New York · Zbl 0277.68056 |
| [14] | Z. Elouedi, P. Vannorenberghe, Inférence statistique et modèle de croyances transférables, in: LFA Toulouse, France, 2006, pp. 61-68.; Z. Elouedi, P. Vannorenberghe, Inférence statistique et modèle de croyances transférables, in: LFA Toulouse, France, 2006, pp. 61-68. |
| [15] | Fonck, P., A comparative study of possibilistic conditional independence and lack of interaction, International Journal of Approximate Reasoning, 16, 149-171 (1997) · Zbl 0939.68115 |
| [16] | Friedman, N.; Geiger, D.; Goldszmidt, M., Bayesian network classifiers, Machine Learning, 29, 131-161 (1997) · Zbl 0892.68077 |
| [17] | Geiger, D.; Heckerman, D., Knowledge representation and inference in similarity networks and Bayesian multinets, Artificial Intelligence, 82, 45-74 (1996) · Zbl 1517.68360 |
| [18] | D. Grossman, P. Dominigos, Learning Bayesian maximizing conditional likelihood, in: Proc. on Machine Learning, Canada, 2004, pp. 46-57.; D. Grossman, P. Dominigos, Learning Bayesian maximizing conditional likelihood, in: Proc. on Machine Learning, Canada, 2004, pp. 46-57. |
| [19] | Inza, I.; Larrañaga, P.; Etxeberria, R.; Sierra, B., Feature subset selection by Bayesian network-based optimization, Artificial Intelligence, 123, 157-184 (2000) · Zbl 0952.68118 |
| [20] | Kohavi, R.; John, G., Wrappers for feature subset selection, Artificial Intelligence Journal, Special Issue on Relevance, 97, 273-324 (1997) · Zbl 0904.68143 |
| [21] | I. Kononenko, Semi-naïve Bayesian classifier, in: Proc. 6th European Working Session on Learning, 1991, pp. 206-219.; I. Kononenko, Semi-naïve Bayesian classifier, in: Proc. 6th European Working Session on Learning, 1991, pp. 206-219. |
| [22] | P. Langley, W. Iba, K. Thompson, An analysis of Bayesian classifiers, in: Proc. AAAI-92, 1992, pp. 223-228.; P. Langley, W. Iba, K. Thompson, An analysis of Bayesian classifiers, in: Proc. AAAI-92, 1992, pp. 223-228. |
| [23] | P. Langley, S. Sage, Induction of selective Bayesian classifiers, in: Proc. 10th Conf. on Uncertainty in Artificial Intelligence UAI-94, 1994, pp. 399-406.; P. Langley, S. Sage, Induction of selective Bayesian classifiers, in: Proc. 10th Conf. on Uncertainty in Artificial Intelligence UAI-94, 1994, pp. 399-406. |
| [24] | X. Li, Augmented naive Bayesian classifiers for mixed-mode data, in: IEEE Workshop on Automatic Speech Recognition and Understanding, Virgin Islands, 2003.; X. Li, Augmented naive Bayesian classifiers for mixed-mode data, in: IEEE Workshop on Automatic Speech Recognition and Understanding, Virgin Islands, 2003. |
| [25] | H. Liu, H. Motoda, Feature Selection, Knowledge Discovery and Data Mining, Kluwer, Boston, 1998.; H. Liu, H. Motoda, Feature Selection, Knowledge Discovery and Data Mining, Kluwer, Boston, 1998. · Zbl 0908.68127 |
| [26] | P.M. Murphy, D.W. Aha, Uci repository of machine learning databases \(\langle;\) http://www.ics.uci.edu/mlearn \(\rangle;\); P.M. Murphy, D.W. Aha, Uci repository of machine learning databases \(\langle;\) http://www.ics.uci.edu/mlearn \(\rangle;\) |
| [27] | M.J. Pazzani, Searching for dependencies in Bayesian classifiers, in: Artificial Intelligence and Statistics IV, Lecture Notes in Statistics, Springer, New York, 1997, pp. 239-248.; M.J. Pazzani, Searching for dependencies in Bayesian classifiers, in: Artificial Intelligence and Statistics IV, Lecture Notes in Statistics, Springer, New York, 1997, pp. 239-248. |
| [28] | J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmman, San Francisco, CA, 1988, pp. 221-226.; J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmman, San Francisco, CA, 1988, pp. 221-226. |
| [29] | Quinlan, J. R., Induction of decision trees, Machine Learning, 1, 81-106 (1986) |
| [30] | M. Sahami, Learning limited dependence Bayesian classifiers, in: Proc. 2nd Int. Conf. On Knowledge Discovery and Data Mining KDD’96, Portland, OR, USA, 1996, pp. 335-338.; M. Sahami, Learning limited dependence Bayesian classifiers, in: Proc. 2nd Int. Conf. On Knowledge Discovery and Data Mining KDD’96, Portland, OR, USA, 1996, pp. 335-338. |
| [31] | Shafer, G., A mathematical Theory of Evidence (1976), Princeton University Press: Princeton University Press Princeton, NJ · Zbl 0359.62002 |
| [32] | Yager, R. R., A general approach to the fusion of imprecise information, International Journal of Intelligent Systems, 12, 1-29 (1997) · Zbl 0869.68101 |
| [33] | Zadeh, L. A., Fuzzy sets, Information and Control, 8, 338-353 (1965) · Zbl 0139.24606 |
| [34] | Zadeh, L. A., Fuzzy sets as a basic for a theory of possibility, Fuzzy Sets and Systems, 1, 3-28 (1978) · Zbl 0377.04002 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
