×

Rough cognitive ensembles. (English) Zbl 1419.68079

Summary: Rough Cognitive Networks are granular classifiers stemming from the hybridization of Fuzzy Cognitive Maps and Rough Set Theory. Such cognitive neural networks attempt to quantify the impact of rough granular constructs (i.e., the positive, negative and boundary regions of a target concept) over each decision class for the problem at hand. In rough classifiers, determining the precise granularity level is crucial to compute high prediction rates. Regrettably, learning the similarity threshold parameter requires reconstructing the information granules, which may be time-consuming. In this paper, we put forth a new multiclassifier system classifier named Rough Cognitive Ensembles. The proposed ensemble employs a collection of Rough Cognitive Networks as base classifiers, each operating at a different granularity level. This allows suppressing the requirement of learning a similarity threshold. We evaluate the granular ensemble with 140 traditional classification datasets using different heterogeneous distance functions. After comparing the proposed model to 15 well-known classifiers, the experimental evidence confirms that our scheme yields very promising classification rates.

MSC:

68T05 Learning and adaptive systems in artificial intelligence
62H30 Classification and discrimination; cluster analysis (statistical aspects)
68T37 Reasoning under uncertainty in the context of artificial intelligence

Software:

UCI-ml; WEKA; C4.5; JStatCom

References:

[1] Duda, R. O.; Hart, P. E.; Stork, D. G., Pattern Classification (2012), John Wiley & Sons
[2] Sun, Y.; Wong, A. K.; Kamel, M. S., Classification of imbalanced data: a review, Int. J. Pattern Recognit. Artif. Intell., 23, 04, 687-719 (2009)
[3] López, V.; Fernández, A.; García, S.; Palade, V.; Herrera, F., An insight into classification with imbalanced data: empirical results and current trends on using data intrinsic characteristics, Inf. Sci., 250, 113-141 (2013)
[4] Frénay, B.; Verleysen, M., Classification in the presence of label noise: a survey, IEEE Trans. Neural Netw. Learn. Syst., 25, 5, 845-869 (2014)
[5] Cohen, I.; Cozman, F. G.; Sebe, N.; Cirelo, M. C.; Huang, T. S., Semisupervised learning of classifiers: theory, algorithms, and their application to human-computer interaction, IEEE Trans. Pattern Anal. Mach. Intell., 26, 12, 1553-1566 (2004)
[6] Tsoumakas, G.; Katakis, I., Multi-label classification: an overview, Int. J. Data Warehous. Min., 3, 3, 1-13 (2007)
[7] Cheng, W.; Hüllermeier, E.; Dembczynski, K. J., Graded multilabel classification: the ordinal case, (Proceedings of the 27th International Conference on Machine Learning. Proceedings of the 27th International Conference on Machine Learning, ICML-10 (2010)), 223-230
[8] Nápoles, G.; Falcon, R.; Papageorgiou, E.; Bello, R.; Vanhoof, K., Partitive granular cognitive maps to graded multilabel classification, (2016 IEEE International Conference on Fuzzy Systems. 2016 IEEE International Conference on Fuzzy Systems, FUZZ-IEEE (2016), IEEE Computational Intelligence Society)
[9] Kotsiantis, S. B., Decision trees: a recent overview, Artif. Intell. Rev., 39, 4, 261-283 (2013)
[10] Zhang, G. P., Neural networks for classification: a survey, IEEE Trans. Syst. Man Cybern., Part C, Appl. Rev., 30, 4, 451-462 (2000)
[11] Ishibuchi, H.; Nakashima, T.; Morisawa, T., Voting in fuzzy rule-based systems for pattern classification problems, Fuzzy Sets Syst., 103, 2, 223-238 (1999)
[12] Friedman, N.; Geiger, D.; Goldszmidt, M., Bayesian network classifiers, Mach. Learn., 29, 2-3, 131-163 (1997) · Zbl 0892.68077
[13] Hearst, M. A.; Dumais, S. T.; Osman, E.; Platt, J.; Scholkopf, B., Support vector machines, IEEE Intell. Syst. Appl., 13, 4, 18-28 (1998)
[14] Cover, T.; Hart, P., Nearest neighbor pattern classification, IEEE Trans. Inf. Theory, 13, 1, 21-27 (1967) · Zbl 0154.44505
[15] Dietterich, T. G., Ensemble methods in machine learning, (Multiple Classifier Systems (2000), Springer), 1-15 · Zbl 0963.68085
[16] Ren, Y.; Zhang, L.; Suganthan, P., Ensemble classification and regression - recent developments, applications and future directions, IEEE Comput. Intell. Mag., 11, 1, 41-53 (2016)
[17] Fan, W.; Stolfo, S. J.; Zhang, J., The application of AdaBoost for distributed, scalable and on-line learning, (Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (1999), ACM), 362-366
[18] Breiman, L., Random forests, Mach. Learn., 45, 1, 5-32 (2001) · Zbl 1007.68152
[19] Zhao, Y.; Gao, J.; Yang, X., A survey of neural network ensembles, (International Conference on Neural Networks and Brain, 2005, vol. 1 (2005), IEEE), 438-442
[20] Yang, P.; Hwa Yang, Y.; Zhou, B. B.; Zomaya, A. Y., A review of ensemble methods in bioinformatics, Current Bioinformatics, 5, 4, 296-308 (2010)
[21] Pal, S. K.; Polkowski, L., Rough-Neural Computing: Techniques for Computing with Words (2004), Springer-Verlag: Springer-Verlag Berlin, Heidelberg · Zbl 1040.68113
[22] Polkowski, L.; Artiemjew, P., On granular rough computing: factoring classifiers through granulated decision systems, (Rough Sets and Intelligent Systems Paradigms (2007), Springer), 280-289
[23] Pedrycz, W.; Park, B.-J.; Oh, S.-K., The design of granular classifiers: a study in the synergy of interval calculus and fuzzy sets in pattern recognition, Pattern Recognit., 41, 12, 3720-3735 (2008) · Zbl 1173.68685
[24] Al-Hmouz, R.; Pedrycz, W.; Balamash, A.; Morfeq, A., From data to granular data and granular classifiers, (2014 IEEE International Conference on Fuzzy Systems. 2014 IEEE International Conference on Fuzzy Systems, FUZZ-IEEE (2014), IEEE), 432-438
[25] Al-Hmouz, R.; Pedrycz, W.; Balamash, A.; Morfeq, A., Description and classification of granular time series, Soft Comput., 19, 4, 1003-1017 (2015)
[26] Szczuka, M.; Jankowski, A.; Skowron, A.; Slezak, D., Building granular systems - from concepts to applications, (Yao, Y.; Hu, Q.; Yu, H.; Grzymala-Busse, J. W., Rough Sets, Fuzzy Sets, Data Mining, and Granular Computing. Rough Sets, Fuzzy Sets, Data Mining, and Granular Computing, Lecture Notes in Computer Science, vol. 9437 (2015), Springer International Publishing), 245-255
[27] Bargiela, A.; Pedrycz, W., Granular Computing: An Introduction, The Springer International Series in Engineering and Computer Science, vol. 717 (2012), Springer Science & Business Media
[28] Polkowski, L.; Artiemjew, P., Granular Computing in Decision Approximation - An Application of Rough Mereology, Intelligent Systems Reference Library, vol. 77 (2015), Springer · Zbl 1314.68006
[29] Pedrycz, W., Granular Computing: Analysis and Design of Intelligent Systems (2013), CRC Press
[30] Nápoles, G.; Grau, I.; Papageorgiou, E.; Bello, R.; Vanhoof, K., Rough cognitive networks, Knowl.-Based Syst., 91, 46-61 (2016)
[31] Kosko, B., Fuzzy cognitive maps, Int. J. Man-Mach. Stud., 24, 1, 65-75 (1986) · Zbl 0593.68073
[32] Pawlak, Z., Rough sets, Int. J. Comput. Inf. Sci., 11, 5, 341-356 (1982) · Zbl 0501.68053
[33] Nápoles, G.; Grau, I.; Vanhoof, K.; Bello, R., Hybrid model based on rough sets theory and fuzzy cognitive maps for decision-making, (Rough Sets and Intelligent Systems Paradigms (2014), Springer), 169-178
[34] Nápoles, G.; Grau, I.; Falcon, R.; Bello, R.; Vanhoof, K., A granular intrusion detection system using rough cognitive networks, (Abielmona, R.; Falcon, R.; Zincir-Heywood, N.; Abbass, H., Recent Advances in Computational Intelligence in Defense and Security (2015), Springer Verlag), Ch. 7
[35] Bello, R.; Falcon, R.; Pedrycz, W.; Kacprzyk, J., Granular Computing: at the Junction of Rough Sets and Fuzzy Sets (2008), Springer Verlag: Springer Verlag Berlin-Heidelberg, Germany · Zbl 1132.68003
[36] Abraham, A.; Falcon, R.; Bello, R., Rough Set Theory: a True Landmark in Data Analysis (2009), Springer Verlag: Springer Verlag Berlin-Heidelberg, Germany · Zbl 1157.68001
[37] Yao, Y., Three-way decision: an interpretation of rules in rough set theory, (Rough Sets and Knowledge Technology (2009), Springer), 642-649
[38] Yao, Y., The superiority of three-way decisions in probabilistic rough set models, Inf. Sci., 181, 6, 1080-1096 (2011) · Zbl 1211.68442
[39] Yao, Y., Three-way decisions with probabilistic rough sets, Inf. Sci., 180, 3, 341-353 (2010)
[40] Kosko, B., Hidden patterns in combined and adaptive knowledge networks, Int. J. Approx. Reason., 2, 4, 377-393 (1988) · Zbl 0655.68108
[41] Bueno, S.; Salmeron, J. L., Benchmarking main activation functions in fuzzy cognitive maps, Expert Syst. Appl., 36, 3, 5221-5229 (2009)
[42] Pedrycz, W.; Homenda, W., From fuzzy cognitive maps to granular cognitive maps, IEEE Trans. Fuzzy Syst., 22, 4, 859-869 (2014)
[43] Pedrycz, W., The design of cognitive maps: a study in synergy of granular computing and evolutionary optimization, Expert Syst. Appl., 37, 10, 7288-7294 (2010)
[44] Pedrycz, W.; Jastrzebska, A.; Homenda, W., Design of fuzzy cognitive maps for modeling time series, IEEE Trans. Fuzzy Syst., 24, 1, 120-130 (2016)
[45] Bezdek, J. C.; Ehrlich, R.; Full, W., FCM: the fuzzy c-means clustering algorithm, Comput. Geosci., 10, 2, 191-203 (1984)
[46] Homenda, W.; Jastrzebska, A.; Pedrycz, W., Granular cognitive maps reconstruction, (2014 IEEE International Conference on Fuzzy Systems. 2014 IEEE International Conference on Fuzzy Systems, FUZZ-IEEE (2014), IEEE), 2572-2579
[47] Bryll, R.; Gutierrez-Osuna, R.; Quek, F., Attribute bagging: improving accuracy of classifier ensembles by using random feature subsets, Pattern Recognit., 36, 6, 1291-1302 (2003) · Zbl 1033.68092
[48] Tumer, K.; Ghosh, J., Classifier combining: analytical results and implications, (Working Notes from the Workshop ‘Integrating Multiple Learned Models’, 13th National Conference on Artificial Intelligence, CiteSeer (1996))
[49] Turner, K.; Oza, N. C., Decimated input ensembles for improved generalization, (International Joint Conference on Neural Networks, 1999, vol. 5. International Joint Conference on Neural Networks, 1999, vol. 5, IJCNN’99 (1999), IEEE), 3069-3074
[50] Breiman, L., Bagging predictors, Mach. Learn., 24, 2, 123-140 (1996) · Zbl 0858.68080
[51] Witten, I. H.; Frank, E., Data Mining: Practical Machine Learning Tools and Techniques (2011), Elsevier
[52] Wilson, D. R.; Martinez, T. R., Improved heterogeneous distance functions, J. Artif. Intell. Res., 6, 1-34 (1997) · Zbl 0894.68118
[53] Quinlan, J., Induction of decision trees, Mach. Learn., 1, 81-106 (1986)
[54] Alcalá, J.; Fernández, A.; Luengo, J.; Derrac, J.; García, S.; Sánchez, L.; Herrera, F., Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework, J. Mult.-Valued Log. Soft Comput., 17, 255-287, 11 (2010)
[55] Lichman, M., UCI machine learning repository (2013)
[56] Hall, M.; Frank, E.; Holmes, G.; Pfahringer, B.; Reutemann, P.; Witten, I. H., The WEKA data mining software: an update, ACM SIGKDD Explor. Newsl., 11, 1, 10-18 (2009)
[57] Triguero, I.; García, S.; Herrera, F., Self-labeled techniques for semi-supervised learning: taxonomy, software and empirical study, Knowl. Inf. Syst., 42, 2, 245-284 (2015)
[58] Kohavi, R., The power of decision tables, (Machine Learning: ECML-95 (1995), Springer), 174-189
[59] John, G. H.; Langley, P., Estimating continuous distributions in Bayesian classifiers, (Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence (1995), Morgan Kaufmann Publishers Inc.), 338-345
[60] Keerthi, S. S.; Shevade, S. K.; Bhattacharyya, C.; Murthy, K. R.K., Improvements to Platt’s SMO algorithm for SVM classifier design, Neural Comput., 13, 3, 637-649 (2001) · Zbl 1085.68629
[61] Hecht-Nielsen, R., Theory of the backpropagation neural network, (International Joint Conference on Neural Networks, 1989. International Joint Conference on Neural Networks, 1989, IJCNN (1989), IEEE), 593-605
[62] Sumner, M.; Frank, E.; Hall, M., Speeding up logistic model tree induction, (Knowledge Discovery in Databases: PKDD 2005 (2005), Springer), 675-683
[63] Quinlan, J. R., C4.5: Programs for Machine Learning (1993), Morgan Kauffman Publishers
[64] Su, J.; Zhang, H., A fast decision tree learning algorithm, (Proceedings of the 21st National Conference on Artificial Intelligence, Vol. 1. Proceedings of the 21st National Conference on Artificial Intelligence, Vol. 1, AAAI’06 (2006), AAAI Press), 500-505
[65] Shi, H., Best-First Decision Tree Learning (2007), CiteSeer, Ph.D. thesis
[66] Landwehr, N.; Hall, M.; Frank, E., Logistic model trees, Mach. Learn., 59, 1-2, 161-205 (2005) · Zbl 1101.68767
[67] Amit, Y.; Geman, D., Shape quantization and recognition with randomized trees, Neural Comput., 9, 7, 1545-1588 (1997)
[68] Aha, D. W.; Kibler, D.; Albert, M. K., Instance-based learning algorithms, Mach. Learn., 6, 1, 37-66 (1991)
[69] Cleary, J. G.; Trigg, L. E., K*: an instance-based learner using an entropic distance measure, (Proceedings of the 12th International Conference on Machine Learning, vol. 5 (1995)), 108-114
[70] Atkeson, C. G.; Moore, A. W.; Schaal, S., Locally weighted learning for control, (Lazy Learning (1997), Springer), 75-113
[71] Smeeton, N. C., Early history of the kappa statistic, Biometrics, 41, 795 (1985)
[72] Friedman, M., The use of ranks to avoid the assumption of normality implicit in the analysis of variance, J. Am. Stat. Assoc., 32, 200, 675-701 (1937) · JFM 63.1098.02
[73] Wilcoxon, F., Individual comparisons by ranking methods, Biometrics, 1, 80-93 (1945)
[74] Benavoli, A.; Corani, G.; Mangili, F., Should we really use post-hoc tests based on mean-ranks?, J. Mach. Learn. Res., 17, 1-10 (2016) · Zbl 1360.62208
[75] Lemke, C.; Budka, M.; Gabrys, B., Metalearning: a survey of trends and technologies, Artif. Intell. Rev., 44, 1, 117-130 (2015)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.