Enhancing the predictive performance of ensemble models through novel multi-objective strategies: evidence from credit risk and business model innovation survey data. (English) Zbl 07712063
Summary: This paper proposes novel multi-objective optimization strategies to develop a weighted ensemble model. The comparison of the performance of the proposed strategies against simulated data suggests that the multi-objective strategy based on joint entropy is superior to other proposed strategies. For the application, generalization, and practical implications of the proposed approaches, we implemented the model on two real datasets related to the prediction of credit risk default and the adoption of the innovative business model by firms. The scope of this paper can be extended in ordering the solutions of the proposed multi-objective strategies and can be generalized for other similar predictive tasks.
Keywords:
multi-objective optimization; ensemble model; prediction; business model innovation; credit riskReferences:
| [1] | Bäck, T. (1996). Evolutionary algorithms in theory and practice: Evolution strategies, evolutionary programming, genetic algorithms. Oxford University Press. · Zbl 0877.68060 |
| [2] | Banner, KM; Higgs, MD, Considerations for assessing model averaging of regression coefficients, Ecological Applications, 27, 78-93 (2017) · doi:10.1002/eap.1419 |
| [3] | Belton, V., & Stewart, T. (2002). Multiple criteria decision analysis: An integrated approach. Springer. |
| [4] | Breskvar, M.; Kocev, D.; Džeroski, S., Ensembles for multi-target regression with random output selections, Machine Learning, 107, 1673-1709 (2018) · Zbl 1475.68262 · doi:10.1007/s10994-018-5744-y |
| [5] | Burnham, K.; Anderson, DR, Model selection and multi-model inference: A practical information-theoretic approach, Springer, 26, 2, 1-488 (2002) · Zbl 1005.62007 |
| [6] | Chawla, Nitesh V.; Bowyer, Kevin W.; Hall, Lawrence O.; Philip Kegelmeyer, W., SMOTE: Synthetic Minority Over-sampling Technique, Journal of Artificial Intelligence Research, 16, 321-57 (2002) · Zbl 0994.68128 · doi:10.1613/jair.953 |
| [7] | Coello, C. A. C., Lamont, G. B., & Van Veldhuizen, D. A. (2007). Evolutionary algorithms for solving multiobjective problems. Springer. · Zbl 1142.90029 |
| [8] | Deb, K. (2001). Multiobjective optimization using evolutionary algorithms (pp. 1-518). Wiley. · Zbl 0970.90091 |
| [9] | Deb, K. (2001). Multiobjective optimization using evolutionary algorithms. Wiley. · Zbl 0970.90091 |
| [10] | Dellnitz, M.; Schutze, O.; Hestermeyer, T., Covering pareto sets by multilevel subdivison techniques, Journal of Optimization Theory and Applications, 124, 1, 113-136 (2005) · Zbl 1137.90015 · doi:10.1007/s10957-004-6468-7 |
| [11] | DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics, 837-845. · Zbl 0715.62207 |
| [12] | Ehrgott, M. (2005). Multicriteria optimization. Springer. · Zbl 1132.90001 |
| [13] | Ehrgott, M., Vilfredo Pareto and multiobjective optimization. Optimization stories, Journal der Deutschen Mathematiker-Vereininggung, Extra, 21, 447-453 (2012) · Zbl 1270.90059 |
| [14] | Fletcher, S.; Verma, B.; Zhang, M., A non-specialized ensemble classifier using multi-objective optimization, Neurocomputing, 409, 93-102 (2020) · doi:10.1016/j.neucom.2020.05.029 |
| [15] | Ignizio, J. P. (1976). Goal programming and extensions. Lexington Books. |
| [16] | Izui, K.; Yamada, T.; Nishiwaki, S.; Tanaka, K., Multiobjective optimization using an aggregative gradient-based method, Structural and Multidisciplinary Optimization, 51, 173-182 (2015) · doi:10.1007/s00158-014-1125-8 |
| [17] | Jin, Y. (2006). Multi-objective machine learning (Vol. 14, pp. 1-660). Springer. · Zbl 1129.68063 |
| [18] | Jürgen, B., Kalyanmoy, D., Kaisa, M., & Roman, S. (2008). Multiobjective optimization: Interactive and evolutionary approaches. Springer. · Zbl 1147.68304 |
| [19] | Kordík, P.; Černý, J.; Frýda, T., Discovering predictive ensembles for transfer learning and meta-learning, Machine Learning, 107, 177-207 (2018) · doi:10.1007/s10994-017-5682-0 |
| [20] | Kou, G.; Peng, Y.; Wang, G., Evaluation of clustering algorithms for financial risk analysis using MCDM methods, Information Sciences, 275, 1-12 (2014) · doi:10.1016/j.ins.2014.02.137 |
| [21] | Kou, G.; Xu, Y.; Peng, Y.; Shen, F.; Chen, Y.; Chang, K.; Kou, S., Bankruptcy prediction for SMEs using transactional data and two-stage multiobjective feature selection, Decision Support Systems, 140 (2021) · doi:10.1016/j.dss.2020.113429 |
| [22] | Kou, G.; Xiao, H.; Cao, M.; Lee, LH, Optimal computing budget allocation for the vector evaluated genetic algorithm in multi-objective simulation optimization, Automatica, 129, 109599 (2021) · Zbl 1480.90220 · doi:10.1016/j.automatica.2021.109599 |
| [23] | Kozodoi, N.; Lessmann, S.; Papakonstantinou, K.; Gatsoulis, Y.; Baesens, B., A multi-objective approach for profit-driven feature selection in credit scoring, Decision Support Systems, 120, 106-117 (2019) · doi:10.1016/j.dss.2019.03.011 |
| [24] | Krawczyk, B., Learning from imbalanced data: Open challenges and future directions, Progress in Artificial Intelligence, 5, 4, 221-232 (2016) · doi:10.1007/s13748-016-0094-0 |
| [25] | Kuncheva, L.; Whitaker, C., Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy”, Machine Learning, 51, 2, 181-207 (2003) · Zbl 1027.68113 · doi:10.1023/A:1022859003006 |
| [26] | Li, T., Kou, G., Peng, Y., & Yu, P. S. (2021). An integrated cluster detection, optimization, and interpretation approach for financial data. IEEE Transactions on Cybernetics, 1-14. |
| [27] | Mackay, D. J. C. (2003). Information theory, inferences, and learning algorithms. Cambridge University Press. · Zbl 1055.94001 |
| [28] | Murphy, K. (2012). Machine learning: A probabilistic perspective. MIT Press. · Zbl 1295.68003 |
| [29] | Peimankar, A.; Weddell, SJ; Jalal, T.; Lapthorn, AC, Multi-objective en- semble forecasting with an application to power transformers, Applied Soft Computing, 68, 233-248 (2018) · doi:10.1016/j.asoc.2018.03.042 |
| [30] | Ribeiro, VHA; Meza, GR, Ensemble learning by means of a multi-objective optimization design approach for dealing with imbalanced data sets, Expert Systems with Applications, 147, 113-232 (2020) |
| [31] | Rosales-Perez, A., Garcia, S., Gonzalez, J. A., Coello, C. A. C., & Herrera, F. (2017). An evolutionary multi-objective model and instance selection for support vector machines with Pareto-based ensembles. IEEE Transactions on Evolutionary Com- putation, 1. |
| [32] | Sahâ, S.; Sarkar, D.; Kramer, S., Exploring multi-objective optimization for multi-label classifier ensembles, IEEE Congress on Evolutionary Computation (CEC), 2019, 2753-2760 (2019) |
| [33] | Shi, C.; Kong, X.; Fu, D.; Yu, PS; Wu, B., Multi-label classification based on multi-objective optimization, Association for Computing Machinery, 5, 2, 1-22 (2014) |
| [34] | Smith, C.; Jin, Y., Evolutionary multi-objective generation of recurrent neu- ral network ensembles for time series prediction, Neurocomputing, 143, 302-311 (2014) · doi:10.1016/j.neucom.2014.05.062 |
| [35] | Tan, CJ; Lim, CP; Cheah, YN, A multi-objective evolutionary algorithm- based ensemble optimizer for feature selection and classification with neural network models, Neurocomputing, 125, 217-228 (2014) · doi:10.1016/j.neucom.2012.12.057 |
| [36] | Tumer, K.; Ghosh, J., Analysis of decision boundaries in linearly combined neural classifiers, Pattern Recognition, 29, 2, 341-348 (1996) · doi:10.1016/0031-3203(95)00085-2 |
| [37] | Wang, W., An effective ensemble framework for multiobjective optimization, IEEE Transactions on Evolutionary Computation, 23, 4, 645-659 (2019) · doi:10.1109/TEVC.2018.2879078 |
| [38] | Wang, F.; Li, Y.; Liao, F.; Yan, H., An ensemble learning based prediction strategy for dynamic multi-objective optimization, Applied Soft Computing, 96, 106592 (2020) · doi:10.1016/j.asoc.2020.106592 |
| [39] | Wozniak, M.; Graña, M.; Corchado, E., A survey of multiple classifier systems as hybrid systems, Information Fusion, 16, 3-17 (2014) · doi:10.1016/j.inffus.2013.04.006 |
| [40] | Zhang, C., & Yunqian, M. (2012). Ensemble machine learning: Methods and applications. Springer. · Zbl 1303.68022 |
| [41] | Zhao, J., Jiao, L., Xia, S., Basto Fernandes, V., Yevseyeva, I., Zhou, Y., Emmerich, T. M., & M. (2018). Multiobjective sparse ensemble learning by means of evolutionary algorithms. Decision Support Systems,111, 86-100. doi:10.1016/j.dss.2018.05.003 |
| [42] | Zhao, H., A multi-objective genetic programming approach to developing Pareto optimal decision trees, Decision Support Systems, 43, 3, 809-826 (2007) · doi:10.1016/j.dss.2006.12.011 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
