A method for improving the accuracy of data mining classification algorithms. (English) Zbl 1160.91325
Summary: We introduce a method called CL.E.D.M. (CLassification through ELECTRE and Data Mining), that employs aspects of the methodological framework of the ELECTRE I outranking method, and aims at increasing the accuracy of existing data mining classification algorithms. In particular, the method chooses the best decision rules extracted from the training process of the data mining classification algorithms, and then it assigns the classes that correspond to these rules, to the objects that must be classified. Three well known data mining classification algorithms are tested in five different widely used databases to verify the robustness of the proposed method.
MSC:
| 91B06 | Decision theory |
| 90B50 | Management decision making, including multiple objectives |
| 62P30 | Applications of statistics in engineering and industry; control charts |
References:
| [1] | Fayyad, U. M.; Piatetsky-Shapiro, G.; Smyth, P., Advances in knowledge discovery and data mining (1996), AAAI Press/MIT Press: AAAI Press/MIT Press Cambridge |
| [2] | Quinlan, J., Induction of decision trees, Machine Learning, 1, 85-106 (1986) |
| [3] | Quinlan, J., C4.5: programs for machine learning (1993), Morgan Kaufmann: Morgan Kaufmann California |
| [4] | Clark, P.; Niblett, T., The CN2 induction algorithm, Machine Learning, 3, 4, 261-283 (1989) |
| [5] | Boutsinas B, Antzoulatos G, Alevizos P. A novel classification algorithm based on clustering. In: First international conference “From scientific computing to computational engineering”, Athens, Greece, 2004.; Boutsinas B, Antzoulatos G, Alevizos P. A novel classification algorithm based on clustering. In: First international conference “From scientific computing to computational engineering”, Athens, Greece, 2004. |
| [6] | Breinman, L.; Friedman, J. H.; Olshen, R. A.; Stone, C. J., Classification and regression trees (1984), Wadsworth and Brooks: Wadsworth and Brooks California · Zbl 0541.62042 |
| [7] | Freitas, A. A., A survey of evolutionary algorithms for data mining and knowledge discovery, (Ghosh, A.; Tsutsui, S., Advances in evolutionary computation (2002), Springer: Springer Berlin) · Zbl 1013.68075 |
| [8] | Friedman, J. H., Multiple adaptive regression splines, Annals of Statistics, 19, 1, 1-141 (1991) · Zbl 0765.62064 |
| [9] | Buntine W. Graphical models for discovering knowledge. In: Fayyad UM, Piatetsky-Shapiro G, Smyth P, editors. Advances in knowledge discovery and data mining. 1996. p. 59-82.; Buntine W. Graphical models for discovering knowledge. In: Fayyad UM, Piatetsky-Shapiro G, Smyth P, editors. Advances in knowledge discovery and data mining. 1996. p. 59-82. |
| [10] | Rumelhart, D. E.; Hinton, G. E.; Williams, R. J., Learning internal representations by error propagation, (Rumelhart, D. E.; McClelland, J. L., Parallel distributed processing: explorations in the microstructure of cognition (1986), MIT Press: MIT Press Cambridge), 318-363 |
| [11] | Cost, S.; Salzberg, S., A weighted nearest neighbour algorithm for learning with symbolic features, Machine Learning, 10, 57-78 (1993) |
| [12] | Dzeroski S. Inductive logic programming and knowledge discovery in databases. In: Fayyad UM, Piatetsky-Shapiro G, Smyth P, editors. Advances in knowledge discovery and data mining. 1996. p. 117-52.; Dzeroski S. Inductive logic programming and knowledge discovery in databases. In: Fayyad UM, Piatetsky-Shapiro G, Smyth P, editors. Advances in knowledge discovery and data mining. 1996. p. 117-52. |
| [13] | Muggleton, S., Inductive logic programming, 38 of A.P.I.C series (1992), Academic Press: Academic Press London |
| [14] | Boutsinas, B.; Vrahatis, M. N., Artificial nonmonotonic neural networks, Artificial Intelligence, 132, 1, 1-38 (2001) · Zbl 0983.68151 |
| [15] | Vapnik, V. N., Statistical learning theory (1998), Wiley: Wiley New York · Zbl 0934.62009 |
| [16] | Vapnik, V. N., The nature of statistical learning theory (2000), Springer: Springer New York · Zbl 0934.62009 |
| [17] | Vovk, V.; Gammerman, A.; Shafer, G., Algorithmic learning in a random world (2005), Springer: Springer New York · Zbl 1105.68052 |
| [18] | Friedman, N.; Geiger, D.; Goldsmidt, M., Bayesian network classifiers, Machine Learning, 29, 2, 131-163 (1997) · Zbl 0892.68077 |
| [19] | Andrews, R.; Diederich, J.; Tickle, A. B., Survey and critique of techniques for extracting rules from trained artificial neural networks, Knowledge-Based Systems, 8, 373-389 (1995) |
| [20] | Quinlan J. Generating production rules from decision trees. In: Proceedings of the 10th IJCAI. 1987. p. 304-7.; Quinlan J. Generating production rules from decision trees. In: Proceedings of the 10th IJCAI. 1987. p. 304-7. |
| [21] | Bouyssou, D., Outranking methods, (Floudas, C. A.; Pardalos, P. M., Encyclopedia of optimization, vol. 4 (2001), Kluwer: Kluwer Dordrecht), 249-255 · Zbl 1027.90001 |
| [22] | Figueira, J.; Mousseau, V.; Roy, B., Electre methods, (Figueira, J.; Greco, S.; Ehrogott, M., Multiple criteria decision analysis: state of the art surveys (2005), Springer: Springer New York), 133-153 · Zbl 1072.90531 |
| [23] | Pirlot, M., A common framework for describing some outranking methods, Journal of Multi-Criteria Decision Analysis, 6, 86-92 (1997) · Zbl 0890.90117 |
| [24] | Roy B, Bertier P. La methode ELECTRE II, une methode de classement en presence de criteres multiples. Note de travail 142. Paris: Direction Scientifique, Sema; 1971.; Roy B, Bertier P. La methode ELECTRE II, une methode de classement en presence de criteres multiples. Note de travail 142. Paris: Direction Scientifique, Sema; 1971. |
| [25] | Roy, B., Classement et choix en présence de points de vue muptiples: la méthode electre, RIRO, 8, 57-75 (1968) |
| [26] | Roy, B., Multicriteria methodology for decision aiding (1996), Kluwer Academic Publishers: Kluwer Academic Publishers Dordrecht · Zbl 0893.90108 |
| [27] | Roy, B., The outranking approach and the foundations of electre methods, Theory and Decision, 31, 49-73 (1991) |
| [28] | Flach, P. A.; Lavrac, N., Rule induction, (Berthold, M.; Hand, D. J., Intelligent data analysis: an introduction (2003), Springer: Springer Berlin), 229-267 |
| [29] | Lavrac N, Flach P, Zupan B. Rule evaluation measures: a unifying view. In: Proceedings of ninth international workshop on inductive logic programming (ILP’99), 1999. p. 174-85.; Lavrac N, Flach P, Zupan B. Rule evaluation measures: a unifying view. In: Proceedings of ninth international workshop on inductive logic programming (ILP’99), 1999. p. 174-85. |
| [30] | Tsumoto S. Characteristics of accuracy and coverage in rule induction. In: Lecture notes in computer science. 2003. p. 237-44.; Tsumoto S. Characteristics of accuracy and coverage in rule induction. In: Lecture notes in computer science. 2003. p. 237-44. · Zbl 1026.68659 |
| [31] | Belacel, N.; Boulassel, M. R., Multicriteria classification fuzzy classification procedure PROCFTN: methodology and medical application, Fuzzy Sets and Systems, 141, 2, 203-217 (2004) · Zbl 1040.92014 |
| [32] | Greco, S.; Matarazzo, B.; Slowinski, R., Rough sets theory for multicriteria decision analysis, European Journal of Operational Research, 129, 1, 1-47 (2001) · Zbl 1008.91016 |
| [33] | Greco, S.; Matarazzo, B.; Slowinski, R., The use of rough sets and fuzzy sets in MCDM, (Gal, T.; Stewart, T.; Hanne, T., Advances in multiple criteria decision making (1999), Kluwer Academic Publishers: Kluwer Academic Publishers Boston, Dordrecht, London), p. 14.1-59 · Zbl 0948.90078 |
| [34] | Slowinski, R.; Stefanowski, J., Rough classification with valued closeness relation, (Diday, E.; etal., New approaches in classification and data analysis (1994), Springer: Springer Berlin), 482-488 |
| [35] | Fayyad UM, Irani KB. Multi-interval discretization of continuous-valued attributes for classification learning. In: Proceedings of international joint conference on artificial intelligence (IJCAI-93), 1993. p. 1022-9.; Fayyad UM, Irani KB. Multi-interval discretization of continuous-valued attributes for classification learning. In: Proceedings of international joint conference on artificial intelligence (IJCAI-93), 1993. p. 1022-9. |
| [36] | Gallier, J. H., Logic for computer science: foundations of automatic theorem proving (2003), Wiley: Wiley New York |
| [37] | Huang, Z., Extensions to the \(k\)-means algorithm for clustering large data sets with categorical values, Data mining and Knowledge Discovery, 2, 59-77 (1998) |
| [38] | Aha D, Murphy P. UCI Repository of machine learning databases. Available at: ⟨http://www.ics.uci.edu/ mlearn/MLRepository.html; Aha D, Murphy P. UCI Repository of machine learning databases. Available at: ⟨http://www.ics.uci.edu/ mlearn/MLRepository.html |
| [39] | Watson, J. D.; Hopkins, N. H.; Roberts, J. W.; Steitz, J. A.; Weiner, A. M., Molecular biology of the gene, vol. 1 (1987), Menlo Park: Menlo Park Benjamin Cummings |
| [40] | Bennett, K. P.; Mangasarian, O. L., Robust linear programming discrimination of two linearly inseparable sets, Optimization Methods and Software, 1, 23-34 (1992) |
| [41] | Bohanec M, Rajkovic V. Knowledge acquisition and explanation for multi-attribute decision making. In: Eighth international workshop on expert systems and their applications, Avignon, France, 1988. p. 59-78.; Bohanec M, Rajkovic V. Knowledge acquisition and explanation for multi-attribute decision making. In: Eighth international workshop on expert systems and their applications, Avignon, France, 1988. p. 59-78. |
| [42] | Shyu ML, Kuruppu-Appuhamilage IP, Chen SC, Chang L. Handling missing values via decomposition of the conditioned set. In: Proceedings of IRI’05, 2005. p. 199-204.; Shyu ML, Kuruppu-Appuhamilage IP, Chen SC, Chang L. Handling missing values via decomposition of the conditioned set. In: Proceedings of IRI’05, 2005. p. 199-204. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
