×

Cost-sensitive business failure prediction when misclassification costs are uncertain: a heterogeneous ensemble selection approach. (English) Zbl 1441.91085

Summary: In order to assess risks associated with establishing relationships with corporate partners such as clients, suppliers, debtors or contractors, decision makers often turn to business failure prediction models. While a large body of literature has focused on optimizing and evaluating novel methods in terms of classification accuracy, recent research has acknowledged the existence of asymmetric misclassification costs associated with prediction errors and thus, advocates the usage of alternative evaluation metrics. However, these papers often assume a misclassification cost matrix to be known and fixed for both the training and the evaluation of models, whereas in reality these costs are often uncertain. This paper presents a methodological framework based upon heterogeneous ensemble selection and multi-objective optimization for cost-sensitive business failure prediction that accommodates uncertainty at the level of misclassification costs. The framework assumes unknown costs during model training and accommodates varying degrees of uncertainty during model deployment. Specifically, NSGA-II is deployed to optimize cost space resulting in a set of Pareto-optimal ensemble classifiers where every learner minimizes expected misclassification cost for a specific range of cost ratios. An extensive set of experiments evaluates the method on multiple data sets and for different scenarios that reflect the extent to which cost ratios are known during model deployment. Results clearly demonstrate the ability of our method to minimize cost under the absence of exact knowledge of misclassification costs.

MSC:

91G45 Financial networks (including contagion, systemic risk, regulation)
62H30 Classification and discrimination; cluster analysis (statistical aspects)
62P05 Applications of statistics to actuarial sciences and financial mathematics
68T05 Learning and adaptive systems in artificial intelligence

Software:

C4.5; AdaCost; NSGA-II; CSMES; R
Full Text: DOI

References:

[1] Abellán, J.; Castellano, J. G., A comparative study on base classifiers in ensemble methods for credit scoring, Expert Systems with Applications, 73, 1-10 (2017)
[2] Adams, N. M.; Hand, D. J., Comparing classifiers when the misallocation costs are uncertain, Pattern Recognition, 32, 7, 1139-1147 (1999)
[3] Altman, E. I., Financial ratios, discriminant analysis and prediction of corporate bankruptcy, Journal of Finance, 23, 4, 589-609 (1968)
[4] Bakker, B.; Heskes, T., Clustering ensembles of neural network models, Neural Networks, 16, 2, 261-269 (2003)
[5] Balcaen, S.; Ooghe, H., 35 years of studies on business failure: an overview of the classic statistical methodologies and their related problems, The British Accounting Review, 38, 1, 63-93 (2006)
[6] Bauer, J.; Agarwal, V., Are hazard models superior to traditional bankruptcy prediction approaches? A comprehensive test, Journal of Banking & Finance, 40, 432-442 (2014)
[7] Bodnar, D., Insolvencies are on the rise in Western Europe (2019), CFO
[8] Bou-Hamad, I.; Larocque, D.; Ben-Ameur, H., Discrete-time survival trees and forests with time-varying covariates: application to bankruptcy data, Statistical Modelling, 11, 5, 429-446 (2011) · Zbl 1420.62417
[9] Breiman, L., Bagging predictors, Machine Learning, 24, 2, 123-140 (1996) · Zbl 0858.68080
[10] Breiman, L., Random forests, Machine Learning, 45, 1, 5-32 (2001) · Zbl 1007.68152
[11] Breiman, L.; Friedman, J. H.; Olsen, R. A.; Stone, C. J., Classification and regression trees (1984), Chapman & Hall/CRC · Zbl 0541.62042
[12] Brigham, E. F.; Gapenski, L. C., Financial management: Theory and practice (1994), The Dryden Press: The Dryden Press Orlando, FL.
[13] Britto, A. S.; Sabourin, R.; Oliveira, L. E., Dynamic selection of classifiers—a comprehensive review, Pattern Recognition, 47, 11, 3665-3680 (2014)
[14] Caruana, R.; Munson, A.; Niculescu-Mizil, A., Getting the most out of ensemble selection, (Proceedings of the 6th international conference on data mining (ICDM ’06) (2006), IEEE Computer Society), 828-833
[15] Caruana, R.; Niculescu-Mizil, A.; Crew, G.; Ksikes, A., Ensemble selection from libraries of models, (Proceedings of the 21st international conference on machine learning (2004), ACM), 18-27
[16] Chatelain, C.; Adam, S.; Lecourtier, Y.; Heutte, L.; Paquet, T., A multi-model selection framework for unknown and/or evolutive misclassification cost problems, Pattern Recognition, 43, 3, 815-823 (2010) · Zbl 1187.68434
[17] Chava, S.; Jarrow, R. A., Bankruptcy prediction with industry effects, Review of Finance, 8, 537-569 (2004) · Zbl 1059.91036
[18] Chen, A.; Chen, N.; Ribeiro, B., Comparative study of classifier ensembles for cost-sensitive credit risk assessment, Intelligent Data Analysis, 19, 1, 127-144 (2015)
[19] Chen, C.; Liaw, A.; Breiman, L., Using random forests to learn imbalanced data (2004), Statistics Department of University of California: Statistics Department of University of California Berkeley, Technical Report 666
[20] Chen, N.; Ribeiro, B., A consensus approach for combining multiple classifiers in cost-sensitive bankruptcy prediction, (Proceedings of the international conference on adaptive and natural computing algorithms (ICANNGA 2013) (2013), Springer-Verlag), pp.
[21] Cheng, F.; Fu, G.; Zhang, X.; Qiu, J., Multi-objective evolutionary algorithm for optimizing the partial area under the ROC curve, Knowledge-Based Systems, 170, 61-69 (2019)
[22] Unternehmensinsolvenzen in Europa - Jahr 2013/14 (2014), Creditreform Wirtschaftsforschung
[23] Croux, C.; Joossens, K.; Lemmens, A., Trimmed bagging, Computational Statistics & Data Analysis, 52, 1, 362-368 (2007) · Zbl 1452.62047
[24] Davalos, S.; Leng, F.; Feroz, E. H.; Cao, Z., Designing an if-then rules-based ensemble of heterogeneous bankruptcy classifiers: a genetic algorithm approach, Intelligent Systems in Accounting, Finance and Management, 21, 3, 129-153 (2014)
[25] De Bock, K. W., Coussement, K., & Lessmann, S. (2020). CSMES: cost-sensitive multi-criteria ensemble selection and other classifiers for cost-sensitive learning under unknown cost conditions (R package version 1.0). Retrieved fromhttps://CRAN.R-project.org/package=CSMES.
[26] Deb, K.; Pratap, A.; Agarwal, S.; Meyarivan, T., A fast and elitist multiobjective genetic algorithm: NSGA-II, IEEE Transactions on Evolutionary Computation, 6, 2, 182-197 (2002)
[27] Demšar, J., Statistical comparisons of classifiers over multiple data sets, Journal of Machine Learning Research, 7, 1-30 (2006) · Zbl 1222.68184
[28] Dimitras, A. I.; Zanakis, S. H.; Zopounidis, C., A survey of business failures with an emphasis on prediction methods and industrial applications, European Journal of Operational Research, 90, 3, 487-513 (1996) · Zbl 0907.90038
[29] Domingos, P., Metacost: a general method for making classifiers cost-sensitive, (Proceedings of the 5th ACM SIGKDD international conference on knowledge discovery and data mining (1999), ACM), 155-164
[30] dos Santos, E. M., Evolutionary algorithms applied to classifier ensemble selection, (Proceedings of the 44th Brazilian operations research symposium/16th Latin Ibero American conference on operations research (2012))
[31] dos Santos, E. M.; Sabourin, R.; Maupin, P., A dynamic overproduce-and-choose strategy for the selection of classifier ensembles, Pattern Recognition, 41, 10, 2993-3009 (2008) · Zbl 1151.68602
[32] Doumpos, M.; Andriosopoulos, K.; Galariotis, E.; Makridou, G.; Zopounidis, C., Corporate failure prediction in the European energy sector: A multicriteria approach and the effect of country characteristics, European Journal of Operational Research, 262, 1, 347-360 (2017)
[33] Doumpos, M.; Zopounidis, C., Model combination for credit risk assessment: A stacked generalization approach, Annals of Operations Research, 151, 1, 289-306 (2007) · Zbl 1132.91488
[34] Drummond, C.; Holte, R., Cost curves: An improved method for visualizing classifier performance, Machine Learning, 65, 1, 95-130 (2006)
[35] Ekinci, A.; Erdal, H.İ, Forecasting bank failure: Base learners, ensembles and hybrid ensembles, Computational Economics, 49, 4, 677-686 (2017)
[36] Fan, W.; Stolfo, S. J.; Zhang, J.; Chan, P. K., AdaCost: Misclassification cost-sensitive boosting, (Proceedings of the sixteenth international conference on machine learning (ICML ’99) (1999), Morgan Kauffman), 97-105
[37] Freund, Y.; Schapire, R. E., Experiments with a new boosting algorithm, (Proceedings of the thirteenth international conference on machine learning (ICML 1996) (1996), Morgan Kauffman: Morgan Kauffman Bari, Italy), 148-156
[38] Friedman, J. H., Stochastic gradient boosting, Computational Statistics & Data Analysis, 38, 4, 367-378 (2002) · Zbl 1072.65502
[39] Friedman, M., The use of ranks to avoid the assumption of normality implicit in the analysis of variance, Journal of the American Statistical Association, 32, 200, 675-701 (1937) · JFM 63.1098.02
[40] Frydman, H.; Altman, E. I.; Kao, D.-L., Introducing recursive partitioning for financial classification: The case of financial distress, Journal of Finance, 40, 1, 269-291 (1985)
[41] García, S.; Fernández, A.; Luengo, J.; Herrera, F., Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power, Information Sciences, 180, 10, 2044-2064 (2010)
[42] Hernández-Orallo, J.; Flach, P.; Ferri, C., A unified view of performance metrics: Translating threshold choice into expected classification loss, Journal of Machine Learning Research, 13, Oct, 2813-2869 (2012) · Zbl 1436.62260
[43] Hernández-Orallo, J.; Flach, P. A.; Ramirez, C. F., Brier curves: A new cost-based visualisation of classifier performance, (Proceedings of the 28th international conference on international conference on machine learning (ICML’11) (2011)), 585-592
[44] Ho, T. K., The random subspace method for constructing decision forests, IEEE Transactions on Pattern Analysis and Machine Intelligence, 20, 8, 832-844 (1998)
[45] Hua, Z.; Wang, Y.; Xu, X.; Zhang, B.; Liang, L., Predicting corporate financial distress based on integration of support vector machine and logistic regression, Expert Systems with Applications, 33, 2, 434-440 (2007)
[46] Jo, H.; Han, I.; Lee, H., Bankruptcy prediction using case-based reasoning, neural networks, and discriminant analysis, Expert Systems with Applications, 13, 2, 97-108 (1997)
[47] Johnson, R. A.; Raeder, T.; Chawla, N. V., Optimizing classifiers for hypothetical scenarios, (Proceedings of the Pacific-Asia conference on knowledge discovery and data mining (PAKDD 2015) (2015), Springer International Publishing), 264-276
[48] Kim, M. H.; Yoo, P. D., A semiparametric model approach to financial bankruptcy prediction, (Proceedings of the 2006 IEEE international conference on engineering of intelligent systems (2006)), 1-6
[49] Kirkos, E., Assessing methodologies for intelligent bankruptcy prediction, Artificial Intelligence Review, 43, 1, 83-123 (2012)
[50] Ko, A. H.R.; Sabourin, R.; Britto, J. A.S., From dynamic classifier selection to dynamic ensemble selection, Pattern Recognition, 41, 5, 1718-1731 (2008) · Zbl 1140.68466
[51] Kolay, M.; Lemmon, M.; Tashjian, E., Spreading the misery? Sources of bankruptcy spillover in the supply chain, Journal of Financial and Quantitative Analysis, 51, 6, 1955-1990 (2016)
[52] Kotsiantis, S.; Tzelepis, D.; Koumanakos, E.; Tampakas, V., Selective costing voting for bankruptcy prediction, International Journal of Knowledge-based and Intelligent Engineering Systems, 11, 2, 115-127 (2007)
[53] Kuncheva, L. I.; Rodriguez, J. J., An experimental study on rotation forest ensembles, (Proceedings of the international workshop on multiple classifier systems (MCS 2007) (2007), Springer-Verlag Berlin: Springer-Verlag Berlin Prague, Czech Republic), 459-468
[54] Lanine, G.; Vennet, R. V., Failure prediction in the Russian bank sector with logit and trait recognition models, Expert Systems with Applications, 30, 3, 463-478 (2006)
[55] Lessmann, S.; Baesens, B.; Seow, H.-V.; Thomas, L. C., Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research, European Journal of Operational Research, 247, 1, 124-136 (2015) · Zbl 1346.90835
[56] Levesque, J.-C.; Durand, A.; Gagne, C.; Sabourin, R., Multi-objective evolutionary optimization for generating ensembles of classifiers in the ROC space, (Proceedings of the 14th annual conference on genetic and evolutionary computation (2012), ACM: ACM Philadelphia, Pennsylvania, USA), 879-886
[57] Li, H.; Sun, J., On performance of case-based reasoning in Chinese business failure prediction from sensitivity, specificity, positive and negative values, Applied Soft Computing, 11, 1, 460-467 (2011)
[58] Li, H.; Sun, J., Principal component case-based reasoning ensemble for business failure prediction, Information & Management, 48, 6, 220-227 (2011)
[59] Li, J., A two-step rejection procedure for testing multiple hypotheses, Journal of Statistical Planning and Inference, 138, 6, 1521-1527 (2008) · Zbl 1131.62067
[60] Li, W.; Ding, S.; Chen, Y.; Yang, S., Heterogeneous ensemble for default prediction of peer-to-peer lending in China, IEEE Access, 6, 54396-54406 (2018)
[61] Lin, F. Y.; McClean, S., A data mining approach to the prediction of corporate failure, Knowledge-Based Systems, 14, 3, 189-195 (2001)
[62] Liu, X.-Y.; Zhou, Z.-H., Learning with cost intervals, (Proceedings of the 16th ACM SIGKDD international conference on knowledge discovery and data mining (2010), ACM: ACM Washington, DC, USA), 403-412
[63] Margineantu, D.; Dietterich, T., Pruning adaptive boosting, (Proceedings of the fourteenth international conference on machine learning (ICML ’97) (1997), Morgan Kaufmann Publishers Inc), 211-218
[64] Martin, D., Early warning of bank failure: A logit regression approach, Journal of Banking and Finance, 1, 3, 249-276 (1977)
[65] Martinez-Munoz, G.; Hernandez-Lobato, D.; Suarez, A., An analysis of ensemble pruning techniques based on ordered aggregation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 31, 2, 245-259 (2009)
[66] McGurr, P. T.; DeVaney, S. A., Predicting business failure of retail firms: An analysis using mixed industry models, Journal of Business Research, 43, 169-176 (1998)
[67] Ohlson; James, A., Financial ratios and the probabilistic prediction of bankruptcy, Journal of Accounting Research, 18, 1, 109 (1980)
[68] Olmeda, I.; Fernández, E., Hybrid classifiers for financial multicriteria decision making: The case of bankruptcy prediction, Computational Economics, 10, 4, 317-335 (1997) · Zbl 0883.90046
[69] Özöğür-Akyüz, S.; Windeatt, T.; Smith, R., Pruning of error correcting output codes by optimization of accuracy-diversity trade off, Machine Learning, 101, 1, 253-269 (2015) · Zbl 1343.68203
[70] Papouskova, M.; Hajek, P., Two-stage consumer credit risk modelling using heterogeneous ensemble learning, Decision Support Systems, 118, 33-45 (2019)
[71] Partalas, I.; Tsoumakas, G.; Vlahavas, I., Pruning an ensemble of classifiers via reinforcement learning, Neurocomputing, 72, 7, 1900-1909 (2009)
[72] Pendharkar, P., Misclassification cost minimizing fitness functions for genetic algorithm-based artificial neural network classifiers, Journal of the Operational Research Society, 60, 8, 1123-1134 (2008) · Zbl 1168.92304
[73] Pendharkar, P. C., A threshold-varying artificial neural network approach for classification and its application to bankruptcy prediction problem, Computers & Operations Research, 32, 10, 2561-2582 (2005) · Zbl 1081.62078
[74] Provost, F.; Domingos, P., Tree induction for probability-based ranking, Machine Learning, 52, 3, 199-215 (2003) · Zbl 1039.68105
[75] Provost, F.; Fawcett, T., Robust Classification for imprecise environments, Machine Learning, 42, 3, 203-231 (2001) · Zbl 0969.68126
[76] Quinlan, R., C4.5: Programs fOR MACHINE LEARning (1993), Morgan Kauffman Publishers: Morgan Kauffman Publishers San Mateo, CA.
[77] Retrieved from https://www.R-project.org.
[78] Ravi Kumar, P.; Ravi, V., Bankruptcy prediction in banks and firms via statistical and intelligent techniques – a review, European Journal of Operational Research, 180, 1, 1-28 (2007) · Zbl 1114.91305
[79] Ravi, V.; Kurniawan, H.; Thai, P. N.K.; Kumar, P. R., Soft computing system for bank performance prediction, Applied Soft Computing, 8, 1, 305-315 (2008)
[80] Rodríguez, J. J.; Kuncheva, L. I.; Alonso, C. J., Rotation forest: A new classifier ensemble method, IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 10, 1619-1630 (2006)
[81] Ross, S. A.; Westerfield, R. W.; Jordan, B. D.; Roberts, G. S., Fundamentals of corporate finance (2002), McGraw-Hill Ryerson: McGraw-Hill Ryerson Toronto, Canada
[82] Sun, J.; Li, H., Listed companies’ financial distress prediction based on weighted majority voting combination of multiple classifiers, Expert Systems with Applications, 35, 3, 818-827 (2008)
[83] Sun, J.; Li, H.; Huang, Q.-H.; He, K.-Y., Predicting financial distress and corporate failure: A review from the state-of-the-art definitions, modeling, sampling, and featuring approaches, Knowledge-Based Systems, 57, 41-56 (2014)
[84] Sun, L.; Shenoy, P. P., Using Bayesian networks for bankruptcy prediction: Some methodological issues, European Journal of Operational Research, 180, 2, 738-753 (2007) · Zbl 1123.90305
[85] Sylvester, J.; Chawla, N. V., Evolutionary ensemble creation and thinning, (Proceedings of the international joint conference on neural networks, 2006. IJCNN’06 (2006), IEEE), 5148-5155
[86] Tsai, C.-F., Feature selection in bankruptcy prediction, Knowledge-Based Systems, 22, 2, 120-127 (2009)
[87] Verikas, A.; Kalsyte, Z.; Bacauskiene, M.; Gelzinis, A., Hybrid and ensemble-based soft computing techniques in bankruptcy prediction: A survey, Soft Computing, 14, 995-1010 (2010)
[88] Viaene, S.; Dedene, G., Cost-sensitive learning and decision making revisited, European Journal of Operational Research, 166, 1, 212-220 (2005) · Zbl 1066.90537
[89] Wang, R., & Tang, K. (2012). Minimax classifier for uncertain costs. arXiv, 1205.0406.
[90] Weiss, G. M., Mining with rarity: A unifying framework, SIGKDD Explorations, 6, 1, 315-354 (2004)
[91] Woloszynski, T.; Kurzynski, M., A probabilistic model of classifier competence for dynamic ensemble selection, Pattern Recognition, 44, 10-11, 2656-2668 (2011) · Zbl 1218.68155
[92] Woloszynski, T.; Kurzynski, M.; Podsiadlo, P.; Stachowiak, G. W., A measure of competence based on random classification for dynamic ensemble selection, Information Fusion, 13, 3, 207-213 (2012)
[93] Xia, Y.; Liu, C.; Da, B.; Xie, F., A novel heterogeneous ensemble credit scoring model based on bstacking approach, Expert Systems with Applications, 93, 182-199 (2018)
[94] Zadrozny, B.; Elkan, C., Learning and making decisions when costs and probabilities are both unknown, (Proceedings of the seventh ACM SIGKDD international conference on knowledge discovery and data mining. Proceedings of the seventh ACM SIGKDD international conference on knowledge discovery and data mining, San Francisco, California (2001), ACM), 204-213
[95] Zadrozny, B.; Elkan, C., Transforming classifier scores into accurate multiclass probability estimates, (Proceedings of the 8th ACM SIGKDD international conference on knowledge discovery and data mining. Proceedings of the 8th ACM SIGKDD international conference on knowledge discovery and data mining, Edmonton, Alberta, Canada (2002), ACM), 694-699
[96] Zhao, J.; Basto Fernandes, V.; Jiao, L.; Yevseyeva, I.; Maulana, A.; Li, R.; Bäck, T.; Tang, K.; T. M. Emmerich, M., Multiobjective optimization of classifiers by means of 3D convex-hull-based evolutionary algorithms, Information Sciences, 367-368, 80-104 (2016)
[97] Zhao, J.; Jiao, L.; Liu, F.; Basto Fernandes, V.; Yevseyeva, I.; Xia, S.; T. M. Emmerich, M., 3D fast convex-hull-based evolutionary multiobjective optimization algorithm, Applied Soft Computing, 67, 322-336 (2018)
[98] Zhou, Z.-H., Ensemble methods: Foundations and algorithms (2012), Chapman & Hall/CRC
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.