A three-way selective ensemble model for multi-label classification. (English) Zbl 1448.68431
Summary: Label ambiguity and data complexity are widely recognized as major challenges in multi-label classification. Existing studies strive to find approximate representations concerning label semantics, however, most of them are predefined, neglecting the personality of instance-label pair. To circumvent this drawback, this paper proposes a three-way selective ensemble (TSEN) model. In this model, three-way decisions is responsible for minimizing uncertainty, whereas ensemble learning is in charge of optimizing label associations. Both label ambiguity and data complexity are firstly reduced, which is realized by a modified probabilistic rough set. For reductions with shared attributes, we further promote the prediction performance by an ensemble strategy. The components in base classifiers are label-specific, and the voting results of instance-based level are utilized for tri-partition. Positive and negative decisions are determined directly, whereas the deferment region is determined by label-specific reduction. Empirical studies on a collection of benchmarks demonstrate that TSEN achieves competitive performance against state-of-the-art multi-label classification algorithms.
MSC:
| 68T37 | Reasoning under uncertainty in the context of artificial intelligence |
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 68T10 | Pattern recognition, speech recognition |
Keywords:
multi-label classification; three-way decisions; selective ensemble; uncertainty; probabilistic rough setReferences:
| [1] | Gibaja, E.; Ventura, S., A tutorial on multilabel learning, ACM Comput. Surv., 47, 3, 1-38 (2015) |
| [2] | Zhang, M. L.; Zhou, Z. H., A review on multi-label learning algorithms, IEEE Trans. Knowl. Data Eng., 26, 8, 1819-1837 (2014) |
| [3] | Zhang, M. L.; Zhou, Z. H., ML-KNN: a lazy learning approach to multi-label learning, Pattern Recognit., 40, 7, 2038-2048 (2007) · Zbl 1111.68629 |
| [4] | Wu, Q. Y.; Tan, M. K.; Song, H. J.; Chen, J.; Ng, M. K., ML-FOREST: a multi-label tree ensemble method for multi-label classification, IEEE Trans. Knowl. Data Eng., 28, 10, 2665-2680 (2016) |
| [5] | Chen, W. J.; Shao, Y. H.; Li, C. N.; Deng, N. Y., MLTSVM: a novel twin support vector machine to multi-label learning, Pattern Recognit., 52, 61-74 (2016) · Zbl 1394.68277 |
| [6] | Tai, F.; Lin, H. T., Multilabel classification with principal label space transformation, Neural Comput., 24, 9, 2508-2542 (2012) · Zbl 1269.68084 |
| [7] | Zhang, Y.; Zhou, Z. H., Multilabel dimensionality reduction via dependence maximization, ACM Trans. Knowl. Discov. Data, 4, 3, 1-21 (2010) |
| [8] | Zhang, M. L.; Wu, L., LIFT: multi-label learning with label-specific features, IEEE Trans. Pattern Anal. Mach. Intell., 37, 1, 107-120 (2015) |
| [9] | Lin, Z. J.; Ding, G. G.; Hu, M. Q.; Wang, J. M., Multi-label classification via feature-aware implicit label space encoding, (International Conference on International Conference on Machine Learning (2014)), 325-333 |
| [10] | Xu, S. P.; Yang, X. B.; Yu, H. L.; Yu, D. J.; Yang, J. Y.; Tsang, E. C.C., Multi-label learning with label-specific feature reduction, Knowl.-Based Syst., 104, 52-61 (2016) |
| [11] | Huang, J.; Li, G. R.; Huang, Q. M.; Wu, X. D., Learning label-specific features and class-dependent labels for multi-label classification, IEEE Trans. Knowl. Data Eng., 28, 12, 3309-3323 (2016) |
| [12] | Gentile, C.; Orabona, F., On multilabel classification and ranking with bandit feedback, J. Mach. Learn. Res., 15, 2451-2487 (2014) · Zbl 1319.62128 |
| [13] | Huang, J.; Li, G. R.; Wang, S. H.; Xue, Z.; Huang, Q. M., Multi-label classification by exploiting local positive and negative pairwise label correlation, Neurocomputing, 257, 164-174 (2017) |
| [14] | Huang, S. J.; Zhou, Z. H., Multi-label learning by exploiting label correlations locally, (Twenty-Sixth AAAI Conference on Artificial Intelligence (2012)), 949-955 |
| [15] | Lo, H. Y.; Lin, S. D.; Wang, H. M., Generalized k-labelsets ensemble for multi-label and cost-sensitive classification, IEEE Trans. Knowl. Data Eng., 26, 7, 1679-1691 (2014) |
| [16] | Tsoumakas, G.; Katakis, I.; Vlahavas, I., Random k-labelsets for multilabel classification, IEEE Trans. Knowl. Data Eng., 23, 7, 1079-1089 (2011) |
| [17] | Kanj, S.; Abdallah, F.; Denoeux, T., Evidential Multi-Label Classification Using the Random k-Label Sets Approach (2012), Springer: Springer Berlin, Heidelberg |
| [18] | Wu, Y. P.; Lin, H. T., Progressive random k-labelsets for cost-sensitive multi-label classification, Mach. Learn., 106, 5, 671-694 (2017) · Zbl 1454.62203 |
| [19] | Weng, W.; Lin, Y. J.; Wu, S. X.; Li, Y. W.; Kang, Y., Multi-label learning based on label-specific features and local pairwise label correlation, Neurocomputing, 273, 385-394 (2017) |
| [20] | Meníca, E. L.; Janssen, F., Learning rules for multi-label classification: a stacking and a separate-and-conquer approach, Mach. Learn., 105, 1, 1-50 (2016) |
| [21] | Zhou, Z. H.; Wu, J.; Tang, W., Ensembling neural networks: many could be better than all, Artif. Intell., 137, 1-2, 239-263 (2002) · Zbl 0995.68077 |
| [22] | Lin, C.; Chen, W. Q.; Qiu, C.; Wu, Y. F.; Krishnan, S.; Zou, Q., LibD3C: ensemble classifiers with a clustering and dynamic selection strategy, Neurocomputing, 123, 424-435 (2014) |
| [23] | Rokach, L.; Schclar, A.; Itach, E., Ensemble methods for multi-label classification, Expert Syst. Appl., 41, 16, 7507-7523 (2014) |
| [24] | Lee, J.; Kim, D. W., Feature selection for multi-label classification using multivariate mutual information, Pattern Recognit. Lett., 34, 3, 349-357 (2013) |
| [25] | Lin, Y. J.; Hu, Q. H.; Liu, J. H.; Chen, J. K.; Duan, J., Multi-label feature selection based on neighborhood mutual information, Appl. Soft Comput., 38, 244-256 (2016) |
| [26] | Li, H.; Li, D. Y.; Zhai, Y. H.; Wang, S. G.; Zhang, J., A novel attribute reduction approach for multi-label data based on rough set theory, Inf. Sci., 367-368, 827-847 (2016) · Zbl 1428.68243 |
| [27] | Peng, H. C.; Long, F. H.; Ding, C., Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy, IEEE Trans. Pattern Anal. Mach. Intell., 27, 8, 1226-1238 (2005) |
| [28] | Lin, Y. J.; Liu, J. H.; Liu, J. H.; Duan, J., Multi-label feature selection based on max-dependency and min-redundancy, Neurocomputing, 168, C, 92-103 (2015) |
| [29] | Lee, J.; Kim, D. W., Memetic feature selection algorithm for multi-label classification, Inf. Sci., 293, 293, 80-96 (2015) |
| [30] | Lee, J.; Kim, D. W., SCLS: multi-label feature selection based on scalable criterion for large label set, Pattern Recognit., 342-352 (2017) |
| [31] | Li, F.; Miao, D. Q.; Pedrycz, W., Granular multi-label feature selection based on mutual information, Pattern Recognit., 67, 410-423 (2017) |
| [32] | Denoeux, T.; Younes, Z.; Abdallah, F., Representing uncertainty on set-valued variables using belief functions, Artif. Intell., 174, 7-8, 479-499 (2010) · Zbl 1209.68542 |
| [33] | Everingham, M.; Gool, L.; Williams, C. K.; Winn, J.; Zisserman, A., The Pascal Visual Object Classes (VOC) challenge, Int. J. Comput. Vis., 88, 2, 303-338 (2010) |
| [34] | Geng, X.; Ji, R. Z., Label distribution learning, IEEE Trans. Knowl. Data Eng., 28, 7, 1734-1748 (2014) |
| [35] | Gao, B. B.; Xing, C.; Xie, C. W.; Wu, J.; Geng, X., Deep label distribution learning with label ambiguity, IEEE Trans. Image Process., 26, 6, 2825-2838 (2017) · Zbl 1409.94163 |
| [36] | Pawlak, Z., Rough sets, Int. J. Comput. Inf. Sci., 341-356 (1982) · Zbl 0501.68053 |
| [37] | Pawlak, Z., Rough Sets: Theoretical Aspects of Reasoning About Data (1992), Kluwer Academic Publishers |
| [38] | Li, H.; Li, D. Y.; Zhai, Y. H.; Wang, S. G.; Zhang, J., A variable precision attribute reduction approach in multilabel decision tables, Sci. World J., 2014, 1-7 (2014) |
| [39] | Yu, Y.; Pedrycz, W.; Miao, D. Q., Neighborhood rough sets based multi-label classification for automatic image annotation, Int. J. Approx. Reason., 54, 9, 1373-1387 (2013) · Zbl 1316.68122 |
| [40] | Vluymans, S.; Cornelis, C.; Herrera, F.; Saeys, Y., Multi-label classification using a fuzzy rough neighborhood consensus, Inf. Sci., 433, 96-114 (2018) · Zbl 1436.68313 |
| [41] | Yao, Y. Y., Three-way decision: an interpretation of rules in rough set theory, (International Conference on Rough Sets and Knowledge Technology (2009)), 642-649 |
| [42] | Yao, Y. Y., The superiority of three-way decisions in probabilistic rough set models, Inf. Sci., 181, 6, 1080-1096 (2011) · Zbl 1211.68442 |
| [43] | Hu, B. Q.; Wong, H.; Yiu, K. F.C., On two novel types of three-way decisions in three-way decision spaces, Int. J. Approx. Reason., 82, 285-306 (2017) · Zbl 1404.68164 |
| [44] | Qian, J.; Dang, C. Y.; Yue, X. D.; Zhang, N., Attribute reduction for sequential three-way decisions under dynamic granulation, Int. J. Approx. Reason., 85, 196-216 (2017) · Zbl 1419.68175 |
| [45] | Li, X. N.; Yi, H. Q.; She, Y. H.; Sun, B. Z., Generalized three-way decision models based on subset evaluation, Int. J. Approx. Reason., 83, C, 142-159 (2017) · Zbl 1404.68168 |
| [46] | Sun, B. Z.; Ma, W. M.; Li, B. J.; Li, X. N., Three-way decisions approach to multiple attribute group decision making with linguistic information-based decision-theoretic rough fuzzy set, Int. J. Approx. Reason., 93, 424-442 (2018) · Zbl 1452.68228 |
| [47] | Read, J.; Pfahringer, B.; Holmes, G., Classifier chains for multi-label classification, Mach. Learn., 85, 333-359 (2011) |
| [48] | Madjarov, G.; Gjorgjevikj, D.; Deroski, S., Two stage architecture for multi-label learning, Pattern Recognit., 45, 3, 1019-1034 (2012) |
| [49] | Kai, W. S.; Chong, H. L., Addressing class-imbalance in multi-label learning via two-stage multi-label hypernetwork, Neurocomputing, 266, 29, 375-389 (2017) |
| [50] | Ren, F. J.; Wang, L., Sentiment analysis of text based on three-way decisions, J. Intell. Fuzzy Syst., 33, 1, 245-254 (2017) · Zbl 1376.68145 |
| [51] | Yao, Y. Y.; Zhao, Y.; Wang, J., On reduct construction algorithms, (Rough Sets & Knowledge Technology Proceedings, vol. 4062 (2006)), 100-117 · Zbl 1154.68513 |
| [52] | Yao, Y. Y.; Wong, S. K., A decision theoretic framework for approximating concepts, Int. J. Man-Mach. Stud., 37, 6, 793-809 (1992) |
| [53] | Guo, Y. W.; Jiao, L. C.; Wang, S.; Wang, S.; Liu, F.; Rong, K. X.; Xiong, T., A novel dynamic rough subspace based selective ensemble, Pattern Recognit., 48, 5, 1638-1652 (2014) |
| [54] | Wang, G. Y.; Ma, X. A.; Yu, H., Monotonic uncertainty measures for attribute reduction in probabilistic rough set model, Int. J. Approx. Reason., 59, C, 41-67 (2015) · Zbl 1328.68231 |
| [55] | Zhang, X. Y.; Miao, D. Q., Three-way attribute reducts, Int. J. Approx. Reason., 88, 401-434 (2017) · Zbl 1418.68217 |
| [56] | Ar, J., Statistical comparisons of classifiers over multiple data sets, J. Mach. Learn. Res., 7, 1, 1-30 (2006) · Zbl 1222.68184 |
| [57] | Tsoumakas, G.; Spyromitros-Xioufis, E.; Vilcek, J.; Vlahavas, I., Mulan: a Java library for multi-label learning, J. Mach. Learn. Res., 12, 7, 2411-2414 (2012) · Zbl 1280.68207 |
| [58] | Read, J.; Reutemann, P.; Pfahringer, B.; Holmes, G., MEKA: a multi-label/multi-target extension to WEKA, J. Mach. Learn. Res., 17, 1, 667-671 (2016) · Zbl 1360.68708 |
| [59] | Kimura, K.; Kudo, M.; Sun, L.; Koujaku, S., Fast random k-labelsets for large-scale multi-label classification, (International Conference on Pattern Recognition (2017)), 438-443 |
| [60] | Tsoumakas, G.; Vlahavas, I., Random k-labelsets: an ensemble method for multilabel classification, (European Conference on Machine Learning. European Conference on Machine Learning, ECML (2007)), 406-417 |
| [61] | Godbole, S.; Sarawagi, S., Discriminative methods for multi-labeled classification, (Advances in Knowledge Discovery and Data Mining. Advances in Knowledge Discovery and Data Mining, Lecture Notes in Computer Science, vol. 3056 (2004)), 22-30 |
| [62] | Schapire, R. E.; Singer, Y., Improved boosting algorithms using confidence-rated predictions, Mach. Learn., 37, 297-336 (1999) · Zbl 0945.68194 |
| [63] | Schapire, R. E.; Singer, Y., BoosTexter: A Boosting-Based Systemfor Text Categorization (2000), Kluwer Academic Publishers · Zbl 0951.68561 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
