Multi-task ordinal regression with labeled and unlabeled data. (English) Zbl 1541.68327
Summary: Ordinal regression (OR) aims to construct the classifier from data with ordered class labels. At present, most of the OR methods consider the OR problem as a single learning task and assume that all samples in the learning task are labeled. Nevertheless, in practice, labeling a large number of samples may be costly. If there are not enough labeled samples available to train the classifier, their classification accuracy may be restricted. To deal with this problem, in this paper, we propose a novel multi-task semi-supervised ordinal regression (MTSSOR) method. Our method is able to incorporate the additional information from the related tasks and unlabeled samples into improving the performance of OR classifiers, when the labeled samples are insufficient. To the best of our knowledge, this is the first attempt on multi-task semi-supervised OR. In the experiments, we compare MTSSOR with the single-task supervised OR methods, single-task semi-supervised OR methods and multi-task supervised OR method on real-world multi-task OR datasets. The mean-zero error (MZE) and mean-absolute error (MAE) are used as evaluation metrics. The experimental results show that compared with these OR methods, MTSSOR can achieve a minimum of 0.015 and up to 0.152 improvements in term of MZE, and a minimum of 0.02 and up to 0.272 improvements in term of MAE.
MSC:
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 62-08 | Computational methods for problems pertaining to statistics |
References:
| [1] | Vega-Márquez, B.; Nepomuceno-Chamorro, I. A.; Rubio-Escudero, C.; Riquelme, J. C., Ocean: ordinal classification with an ensemble approach, Inf. Sci., 580, 221-242 (2021) |
| [2] | He, F.; Zeng, Y.; Zheng, L.; Wu, Q., Optimality of regularized least squares ranking with imperfect kernels, Inf. Sci., 589, 564-579 (2022) · Zbl 07831034 |
| [3] | Tian, Q.; Chen, S.; Qiao, L., Ordinal margin metric learning and its extension for cross-distribution image data, Inf. Sci., 349, 50-64 (2016) · Zbl 1398.68461 |
| [4] | Kramer, S.; Widmer, G.; Pfahringer, B.; De Groeve, M., Prediction of ordinal classes using regression trees, Fundam. Inform., 47, 1-2, 1-13 (2001) · Zbl 1016.68079 |
| [5] | Sánchez-Monedero, J.; Gutiérrez, P. A.; Tiňo, P.; Hervás-Martínez, C., Exploitation of pairwise class distances for ordinal classification, Neural Comput., 25, 9, 2450-2485 (2013) · Zbl 1448.91247 |
| [6] | Liu, J.; Sui, C.; Deng, D.; Wang, J.; Feng, B.; Liu, W.; Wu, C., Representing conditional preference by boosted regression trees for recommendation, Inf. Sci., 327, 1-20 (2016) |
| [7] | Chu, W.; Keerthi, S. S., Support vector ordinal regression, Neural Comput., 19, 3, 792-815 (2007) · Zbl 1127.68080 |
| [8] | Zhao, B.; Wang, F.; Zhang, C., Block-quantized support vector ordinal regression, IEEE Trans. Neural Netw., 20, 5, 882-890 (2009) |
| [9] | Liao, H.; Wu, J.; Mao, Y.; Zhou, M.; Vidmer, A.; Lu, K., Addressing time bias in bipartite graph ranking for important node identification, Inf. Sci., 540, 38-50 (2020) |
| [10] | Gu, B.; Sheng, V. S.; Tay, K. Y.; Romano, W.; Li, S., Incremental support vector learning for ordinal regression, IEEE Trans. Neural Netw. Learn. Syst., 26, 7, 1403-1416 (2014) |
| [11] | Wang, H.; Shi, Y.; Niu, L.; Tian, Y., Nonparallel support vector ordinal regression, IEEE Trans. Cybern., 47, 10, 3306-3317 (2017) |
| [12] | Gu, B., A regularization path algorithm for support vector ordinal regression, Neural Netw., 98, 114-121 (2018) · Zbl 1434.68410 |
| [13] | Gao, Y.; Zhao, L., Incomplete label multi-task ordinal regression for spatial event scale forecasting, (Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)) |
| [14] | Wang, L.; Zhu, D., Tackling ordinal regression problem for heterogeneous data: sparse and deep multi-task learning approaches, Data Min. Knowl. Discov., 35, 3, 1134-1161 (2021) · Zbl 1478.62190 |
| [15] | Obozinski, G.; Taskar, B.; Jordan, M., Multi-task feature selection (2006), Statistics Department, UC Berkeley, Tech. Rep. |
| [16] | Argyriou, A.; Evgeniou, T.; Pontil, M., Convex multi-task feature learning, Mach. Learn., 73, 3, 243-272 (2008) · Zbl 1470.68073 |
| [17] | Liu, Y.; Huang, L.; Li, J.; Zhang, W.; Sheng, Y.; Wei, Z., Multi-task learning based on geometric invariance discriminative features, Appl. Intell., 53, 3, 3505-3518 (2023) |
| [18] | Lin, J.; Chen, Q.; Xue, B.; Zhang, M., Multi-task optimisation for multi-objective feature selection in classification, (Fieldsend, J. E.; Wagner, M., GECCO ’22: Genetic and Evolutionary Computation Conference. GECCO ’22: Genetic and Evolutionary Computation Conference, Companion Volume, Boston, Massachusetts, USA, July 9-13, 2022 (2022), ACM), 264-267 |
| [19] | Chang, W.; Nie, F.; Wang, R.; Li, X., Calibrated multi-task subspace learning via binary group structure constraint, Inf. Sci., 631, 271-287 (2023) · Zbl 07829635 |
| [20] | Evgeniou, T.; Pontil, M., Regularized multi-task learning, (Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2004)), 109-117 |
| [21] | Parameswaran, S.; Weinberger, K. Q., Large margin multi-task metric learning, Adv. Neural Inf. Process. Syst., 23, 1867-1875 (2010) |
| [22] | An, R.; Xu, Y.; Liu, X., Multi-task twin bounded support vector machine and its safe screening rule, Appl. Soft Comput., 138, Article 110188 pp. (2023) |
| [23] | Wang, T.; Xu, Y.; Liu, X., Multi-task twin spheres support vector machine with maximum margin for imbalanced data classification, Appl. Intell., 53, 3, 3318-3335 (2023) |
| [24] | Liu, B.; Chen, Q.; Xiao, Y.; Wang, K.; Liu, J.; Huang, R.; Li, L., Semi-supervised multi-task learning with auxiliary data, Inf. Sci., 626, 626-639 (2023) |
| [25] | Williams, C.; Bonilla, E. V.; Chai, K. M., Multi-task Gaussian process prediction, Adv. Neural Inf. Process. Syst., 153-160 (2007) |
| [26] | Zhang, Y.; Yeung, D.-Y., A convex formulation for learning task relationships in multi-task learning, (Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence (2010)), 733-742 |
| [27] | Yu, S.; Yu, K.; Tresp, V.; Kriegel, H.-P., Collaborative ordinal regression, (Proceedings of the 23rd International Conference on Machine Learning (2006)), 1089-1096 |
| [28] | Liu, Y.; Liu, Y.; Zhong, S.; Chan, K. C., Semi-supervised manifold ordinal regression for image ranking, (Proceedings of the 19th ACM International Conference on Multimedia (2011)), 1393-1396 |
| [29] | Seah, C.-W.; Tsang, I. W.; Ong, Y.-S., Transductive ordinal regression, IEEE Trans. Neural Netw. Learn. Syst., 23, 7, 1074-1086 (2012) |
| [30] | Srijith, P.; Shevade, S.; Sundararajan, S., Semi-supervised Gaussian process ordinal regression, (Joint European Conference on Machine Learning and Knowledge Discovery in Databases (2013), Springer), 144-159 |
| [31] | Wu, Y.; Sun, Y.; Liang, X.; Tang, K.; Cai, Z., Evolutionary semi-supervised ordinal regression using weighted kernel Fisher discriminant analysis, (2015 IEEE Congress on Evolutionary Computation (CEC) (2015), IEEE), 3279-3286 |
| [32] | Pérez-Ortiz, M.; Gutiérrez, P. A.; Carbonero-Ruz, M.; Hervás-Martínez, C., Semi-supervised learning for ordinal kernel discriminant analysis, Neural Netw., 84, 57-66 (2016) · Zbl 1428.68251 |
| [33] | Shi, W.; Gu, B.; Li, X.; Huang, H., Quadruply stochastic gradient method for large scale nonlinear semi-supervised ordinal regression AUC optimization, (Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34 (2020)), 5734-5741 |
| [34] | Tsuchiya, T.; Charoenphakdee, N.; Sato, I.; Sugiyama, M., Semisupervised ordinal regression based on empirical risk minimization, Neural Comput., 33, 12, 3361-3412 (2021) · Zbl 1522.68485 |
| [35] | Chen, H.; Jia, Y.; Ge, J.; Gu, B., Incremental learning algorithm for large-scale semi-supervised ordinal regression, Neural Netw., 149, 124-136 (2022) · Zbl 07750027 |
| [36] | Vapnik, V. N., Statistical Learning Theory (1998), Wiley-Interscience · Zbl 0935.62007 |
| [37] | Bennett, K. P.; Demiriz, A., Semi-supervised support vector machines, (Advances in Neural Information Processing Systems (1999)), 368-374 |
| [38] | Joachims, T., Transductive inference for text classification using support vector machines, (Proceedings of the Sixteenth International Conference on Machine Learning (1999)), 200-209 |
| [39] | Li, Y.-F.; Zhou, Z.-H., Towards making unlabeled data never hurt, IEEE Trans. Pattern Anal. Mach. Intell., 37, 1, 175-188 (2015) |
| [40] | Belkin, M.; Niyogi, P.; Sindhwani, V., Manifold regularization: a geometric framework for learning from labeled and unlabeled examples, J. Mach. Learn. Res., 7, 11 (2006) · Zbl 1222.68144 |
| [41] | Golub, G. H.; Van Loan, C. F., Matrix Computations (2013), JHU Press · Zbl 1268.65037 |
| [42] | Sun, B.-Y.; Li, J.; Wu, D. D.; Zhang, X.-M.; Li, W.-B., Kernel discriminant learning for ordinal regression, IEEE Trans. Knowl. Data Eng., 22, 6, 906-910 (2009) |
| [43] | Wang, M.; Yang, L.; Hua, X.-S., MSRA-MM: bridging research and industrial societies for multimedia information retrieval (March 2009), Tech. Rep. MSR-TR-2009-30 |
| [44] | Zhang, J.; Huan, J., Inductive multi-task learning with multiple view data, (Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2012)), 543-551 |
| [45] | Mueller, S. G.; Weiner, M. W.; Thal, L. J.; Petersen, R. C.; Jack, C.; Jagust, W.; Trojanowski, J. Q.; Toga, A. W.; Beckett, L., The Alzheimer’s disease neuroimaging initiative, Neuroimaging Clin. N. Am., 15, 4, 869-877 (2005) |
| [46] | Liang, L.; Lin, L.; Jin, L.; Xie, D.; Li, M., SCUT-FBP5500: a diverse benchmark dataset for multi-paradigm facial beauty prediction, (2018 24th International Conference on Pattern Recognition (ICPR) (2018), IEEE), 1598-1603 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
