On the confusion matrix in credit scoring and its analytical properties. (English) Zbl 1511.91160
Summary: Confusion Matrix is an important measure to evaluate the accuracy of credit scoring models. However, the literature about Confusion Matrix is limited. The analytical properties of Confusion Matrix are ignored. Moreover, the concept of Confusion Matrix is confusing. In this article, we systematically study Confusion Matrix and its analytical properties. We enumerate 16 possible variants of Confusion Matrix and show that only 8 are reasonable. We study the relationship between Confusion Matrix and 2 other performance measures: the receiver operating characteristic curve (ROC) and Kolmogorov-Smirnov statistic (KS). We show that an optimal cutoff score can be attained by KS.
MSC:
| 91G40 | Credit risk |
| 62P05 | Applications of statistics to actuarial sciences and financial mathematics |
Keywords:
confusion matrix; sensitivity; specificity; ROC; KS; false positive; false negative; true positive; true negativeReferences:
| [1] | Huang, J. (2014) |
| [2] | Kohavi, R.; Provost, F., Editorial for the special issue on applications of machine learning and the knowledge discovery process, On applied research in machine learning (1998), New York: Columbia University, New York |
| [3] | Lantz, B., Machine learning with R (2015), Birmingham: PACKT Publishing, Birmingham |
| [4] | Louzada, F.; Ara, A.; Fernandes, G. B., Classification methods applied to credit scoring: A systematic review and overall comparison, Surveys in Operations Research and Management Science, 21, 2, 117-134 (2016) |
| [5] | Mailund, T. (2017) |
| [6] | Refaat, M., Credit risk scorecards: Development and implementation using SAS (2011), Raleigh, NC: LULU.COM, Raleigh, NC |
| [7] | Siddiqi, N., Credit risk scorecards: Developing and implementing intelligent credit scoring (2006), Hoboken, New Jersey: John Wiley & Sons, Inc, Hoboken, New Jersey |
| [8] | Thomas, L. C., Consumer credit models: Pricing, profit and portfolios (2009), Oxford: Oxford University Press, Oxford |
| [9] | Thomas, L. C.; Edelman, D. B.; Crook, J. N., Credit scoring and its applications (2002), Philadelphia, PA: Society for Industrial and Applied Mathematics, Philadelphia, PA · Zbl 1001.91052 |
| [10] | Zeng, G., Metric divergence measures and information value in credit scoring, Journal of Mathematics, 2013, 1 (2013) · Zbl 1486.94043 · doi:10.1155/2013/848271 |
| [11] | Zeng, G., A rule of thumb for reject inference in credit scoring, Mathematical Finance Letters, 2014, 1-13 (2014) |
| [12] | Zeng, G., A necessary condition for a good binning algorithm in credit scoring, Applied Mathematical Sciences Applied Sciences, 8, 65, 3229-42 (2014) · doi:10.12988/ams.2014.44300 |
| [13] | Zeng, G., A unified definition of mutual information with applications in machine learning, Mathematical Problems in Engineering, 2015, 1 (2015) · Zbl 1395.94204 · doi:10.1155/2015/201874 |
| [14] | Zeng, G., A comparison study of computational methods of Kolmogorov-Smirnov statistic in credit scoring, Communications in Statistics: Simulation and Computation, 46, 10, 7744-60 (2017) · Zbl 1383.62257 · doi:10.1080/03610918.2016.1249883 |
| [15] | Zeng, G., Invariant properties of logistic regression model in credit scoring under monotonic transformations, Communications in Statistics: Theory and Methods, 46, 17, 8791-807 (2017) · Zbl 1377.62153 · doi:10.1080/03610926.2016.1193200 |
| [16] | Zeng, G., On the existence of maximum likelihood estimates for weighted logistic regression, Communications in Statistics: Theory and Methods, 46, 22, 11194-203 (2017) · Zbl 1379.62050 · doi:10.1080/03610926.2016.1260742 |
| [17] | Zeng, G.; Zeng, E., On the three-way equivalence of AUC in credit scoring with tied scores, Communications in Statistics: Theory and Methods (2018) · Zbl 07530841 · doi:10.1080/03610926.2018.1435814 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
