Granular maximum decision entropy-based monotonic uncertainty measure for attribute reduction. (English) Zbl 1452.68210
Summary: Attribute reduction is considered an important preprocessing step in machine learning, pattern recognition, and data mining, and several attribute reduction measures based on rough set theory have been introduced to deal with vague, imprecise, or uncertain data. However, some of the measures inherently suffer from nonmonotonicity and redundancy. In this paper, a monotonic uncertainty measure, called granular maximum decision entropy (GMDE), is proposed. Specifically, we first develop a notion of maximum decision entropy. By integrating the uncertainty of the maximum decision entropy with the granulation of knowledge, a novel uncertainty measure is then presented, and its monotonicity is theoretically proved. We also provide a forward heuristic attribute reduction algorithm based on the proposed uncertainty measure, which could simultaneously select the informative attributes and remove the unnecessary attributes in the procedure of attribute reduction, thus resulting in high efficiency. The experiments conducted on several UCI data sets demonstrate that the proposed measure and algorithm are effective and computationally inexpensive and are superior to the representatives in terms of classification performance and efficiency.
MSC:
| 68T37 | Reasoning under uncertainty in the context of artificial intelligence |
Keywords:
rough set theory; attribute reduction; uncertainty measure; maximum decision; knowledge granularityReferences:
| [1] | Li, Y.; Li, T.; Liu, H., Recent advances in feature selection and its applications, Knowl. Inf. Syst., 53, 551-577 (2017) |
| [2] | Bishop, C., Pattern Recognition and Machine Learning (2007), Springer: Springer New York, NY, USA |
| [3] | Armanfard, N.; Reilly, J. P.; Komeili, M., Local feature selection for data classification, IEEE Trans. Pattern Anal. Mach. Intell., 38, 1217-1227 (2016) |
| [4] | Miao, D. Q.; Zhao, Y.; Yao, Y. Y.; Li, H. X.; Xu, F. F., Relative reducts in consistent and inconsistent decision tables of the Pawlak rough set model, Inf. Sci., 179, 4140-4150 (2009) · Zbl 1183.68608 |
| [5] | Li, F.; Miao, D. Q.; Pedrycz, W., Granular multi-label feature selection based on mutual information, Pattern Recognit., 67, 410-423 (2017) |
| [6] | Yao, Y. Y.; Zhao, Y., Discernibility matrix simplification for constructing attribute reducts, Inf. Sci., 179, 867-882 (2009) · Zbl 1162.68704 |
| [7] | Lai, Z. H.; Xu, Y.; Yang, J.; Shen, L. L.; Zhang, D., Rotational invariant dimensionality reduction algorithms, IEEE Trans. Cybern., 47, 3733-3746 (2017) |
| [8] | Pawlak, Z., Rough sets, Int. J. Comput. Inf. Sci., 11, 341-356 (1982) · Zbl 0501.68053 |
| [9] | Pawlak, Z., Rough Sets: Theoretical Aspects of Reasoning About Data (1991), Kluwer Academic Publishers: Kluwer Academic Publishers Dordrecht, Netherlands · Zbl 0758.68054 |
| [10] | Thangavel, K.; Pethalakshmi, A., Dimensionality reduction based on rough set theory: a review, Appl. Soft Comput., 9, 1-12 (2009) |
| [11] | Zhang, X. Y.; Miao, D. Q., Three-way attribute reducts, Int. J. Approx. Reason., 88, 401-434 (2017) · Zbl 1418.68217 |
| [12] | Jia, X. Y.; Shang, L.; Zhou, B.; Yao, Y. Y., Generalized attribute reduct in rough set theory, Knowl.-Based Syst., 91, 204-218 (2016) |
| [13] | Hu, Q. H.; Pedrycz, W.; Yu, D. R.; Lang, J., Selecting discrete and continuous features based on neighborhood decision error minimization, IEEE Trans. Syst. Man Cybern., Part B, Cybern., 40, 137-150 (2010) |
| [14] | Dai, J. H.; Hu, Q. H.; Zhang, J. H.; Hu, H.; Zheng, N. G., Attribute selection for partially labeled categorical data by rough set approach, IEEE Trans. Cybern., 47, 2460-2471 (2017) |
| [15] | Pawlak, Z.; Wong, S. K.M.; Ziarko, W., Rough sets: probabilistic versus deterministic approach, Int. J. Man-Mach. Stud., 29, 81-95 (1988) · Zbl 0663.68094 |
| [16] | Hu, X. H.; Cercone, N., Learning in relational databases: a rough set approach, Comput. Intell., 11, 323-338 (1995) |
| [17] | Skowron, A.; Rauszer, C., The discernibility matrices and functions in information systems, (Słowiński, R., Intelligent Decision Support: Handbook of Applications and Advances of the Rough Sets Theory (1992), Springer: Springer Netherlands, Dordrecht), 331-362 · Zbl 0820.68001 |
| [18] | Ziarko, W., Variable precision rough set model, J. Comput. Syst. Sci., 46, 39-59 (1993) · Zbl 0764.68162 |
| [19] | Zhang, W. X.; Mi, J. S.; Wu, W. Z., Approaches to knowledge reductions in inconsistent systems, Int. J. Intell. Syst., 18, 989-1000 (2003) · Zbl 1069.68606 |
| [20] | Mi, J. S.; Wu, W. Z.; Zhang, W. X., Approaches to knowledge reduction based on variable precision rough set model, Inf. Sci., 159, 255-272 (2004) · Zbl 1076.68089 |
| [21] | Yao, Y. Y.; Zhao, Y., Attribute reduction in decision-theoretic rough set models, Inf. Sci., 178, 3356-3373 (2008) · Zbl 1156.68589 |
| [22] | Li, H. X.; Zhou, X. Z.; Zhao, J. B.; Liu, D., Non-monotonic attribute reduction in decision-theoretic rough sets, Fundam. Inform., 126, 415-432 (2013) · Zbl 1275.68147 |
| [23] | Zhang, X. Y.; Miao, D. Q., Region-based quantitative and hierarchical attribute reduction in the two-category decision theoretic rough set model, Knowl.-Based Syst., 71, 146-161 (2014) |
| [24] | Zhang, X. Y.; Miao, D. Q., Reduction target structure-based hierarchical attribute reduction for two-category decision-theoretic rough sets, Inf. Sci., 277, 755-776 (2014) · Zbl 1354.68268 |
| [25] | Jia, X. Y.; Liao, W. H.; Tang, Z. M.; Shang, L., Minimum cost attribute reduction in decision-theoretic rough set models, Inf. Sci., 219, 151-167 (2013) · Zbl 1293.91049 |
| [26] | Jia, X. Y.; Tang, Z. M.; Liao, W. H.; Shang, L., On an optimization representation of decision-theoretic rough set model, Int. J. Approx. Reason., 55, 156-166 (2014) · Zbl 1316.68179 |
| [27] | Liao, S. J.; Zhu, Q. X.; Min, F., Cost-sensitive attribute reduction in decision-theoretic rough set models, Math. Probl. Eng., 35, 1-9 (2014) · Zbl 1407.68486 |
| [28] | Miao, D. Q.; Hu, G. R., A heuristic algorithm for reduction of knowledge, J. Comput. Res. Dev., 36, 681-684 (1999), (in Chinese) |
| [29] | Lin, Y. J.; Hu, Q. H.; Liu, J. H.; Chen, J. K.; Duan, J., Multi-label feature selection based on neighborhood mutual information, Appl. Soft Comput., 38, 244-256 (2016) |
| [30] | Wang, G. Y.; Yu, H.; Yang, D. C., Decision table reduction based on conditional information entropy, Chinese J. Comput., 25, 759-766 (2002), (in Chinese) |
| [31] | Ma, X. A.; Wang, G. Y.; Yu, H.; Li, T. R., Decision region distribution preservation reduction in decision-theoretic rough set model, Inf. Sci., 278, 614-640 (2014) · Zbl 1354.68264 |
| [32] | Qian, Y. H.; Liang, X. Y.; Lin, G. P.; Guo, Q.; Liang, J. Y., Local multigranulation decision-theoretic rough sets, Int. J. Approx. Reason., 82, 119-137 (2017) · Zbl 1404.68172 |
| [33] | Qian, J.; Dang, C. Y.; Yue, X. D.; Zhang, N., Attribute reduction for sequential three-way decisions under dynamic granulation, Int. J. Approx. Reason., 85, 196-216 (2017) · Zbl 1419.68175 |
| [34] | Zhou, J.; Lai, Z. H.; Miao, D. Q.; Gao, C.; Yue, X. D., Multigranulation rough-fuzzy clustering based on shadowed sets, Inf. Sci. (2018) |
| [35] | Li, W. W.; Huang, Z. Q.; Jia, X. Y.; Cai, X. Y., Neighborhood based decision-theoretic rough set models, Int. J. Approx. Reason., 69, 1-17 (2016) · Zbl 1344.68238 |
| [36] | Yue, X. D.; Chen, Y. F.; Miao, D. Q.; Qian, J., Tri-partition neighborhood covering reduction for robust classification, Int. J. Approx. Reason., 83, 371-384 (2017) · Zbl 1404.68125 |
| [37] | Liu, D.; Liang, D. C.; Wang, C. C., A novel three-way decision model based on incomplete information system, Knowl.-Based Syst., 91, 32-45 (2016) |
| [38] | Jing, Y. G.; Li, T. R.; Fujita, H.; Yu, Z.; Wang, B., An incremental attribute reduction approach based on knowledge granularity with a multi-granulation view, Inf. Sci., 411, 23-38 (2017) |
| [39] | Zhou, J.; Lai, Z. H.; Gao, C.; Miao, D. Q.; Yue, X. D., Rough possibilistic C-means clustering based on multigranulation approximation regions and shadowed sets, Knowl.-Based Syst., 160, 144-166 (2018) |
| [40] | Li, J. H.; Kumar, C. A.; Mei, C. L.; Wang, X. Z., Comparison of reduction in formal decision contexts, Int. J. Approx. Reason., 80, 100-122 (2017) · Zbl 1400.68208 |
| [41] | Yao, Y. Y.; Wong, S. K.M., A decision theoretic framework for approximating concepts, Int. J. Man-Mach. Stud., 37, 793-809 (1992) |
| [42] | Yao, Y. Y., Probabilistic rough set approximations, Int. J. Approx. Reason., 49, 255-271 (2008) · Zbl 1191.68702 |
| [43] | Yao, Y. Y., Three-way decisions with probabilistic rough sets, Inf. Sci., 180, 341-353 (2010) |
| [44] | Yao, Y. Y., The superiority of three-way decisions in probabilistic rough set models, Inf. Sci., 181, 1080-1096 (2011) · Zbl 1211.68442 |
| [45] | Yao, Y. Y., Three-way decisions and cognitive computing, Cogn. Comput., 8, 543-554 (2016) |
| [46] | Liu, C. H.; Pedrycz, W.; Jiang, F.; Wang, M. Z., Decision-theoretic rough set approaches to multi-covering approximation spaces based on fuzzy probability measure, J. Intell. Fuzzy Syst., 34, 1917-1931 (2018) |
| [47] | Liang, J. Y.; Shi, Z. Z., The information entropy, rough entropy and knowledge granulation in rough set theory, Int. J. Uncertain. Fuzziness Knowl.-Based Syst., 12, 37-46 (2004) · Zbl 1074.68072 |
| [48] | Yao, J. T.; Vasilakos, A. V.; Pedrycz, W., Granular computing: perspectives and challenges, IEEE Trans. Cybern., 43, 1977-1989 (2013) |
| [49] | John, G. H.; Kohavi, R.; Pfleger, K., Irrelevant features and the subset selection problem, (Proceedings of the 11th International Conference on Machine Learning. Proceedings of the 11th International Conference on Machine Learning, San Francisco, CA, USA (1994)), 121-129 |
| [50] | Lichman, M., UCI Machine Learning Repository (2013), University of California, School of Information and Computer Science: University of California, School of Information and Computer Science Irvine, CA, USA |
| [51] | Frank, E.; Hall, M. A.; Witten, I. H., The WEKA Workbench, Online Appendix for “Data Mining: Practical Machine Learning Tools and Techniques” (2016), Morgan Kaufmann |
| [52] | Miao, D. Q.; Gao, C.; Zhang, N.; Zhang, Z. F., Diverse reduct subspaces based co-training for partially labeled data, Int. J. Approx. Reason., 52, 1103-1117 (2011) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
