×

Analogy-based classifiers for nominal or numerical data. (English) Zbl 1419.68073

Summary: Introduced a decade ago, analogy-based classification methods constitute a noticeable addition to the set of instance-based learning techniques. They provide valuable results in terms of accuracy on many classical datasets. They rely on the notion of analogical proportions which are statements of the form “A is to B as C is to D”. Analogical proportions have been in particular formalized in Boolean and numerical settings. In both cases, one of the four components of the proportion can be computed from the three others, when the proportion holds. Analogical classifiers look for all triples of examples in the sample set that are in analogical proportion with the item to be classified on a maximal number of attributes and for which the corresponding analogical proportion equation on the class has a solution. In this paper when classifying a new item, we specially emphasize an approach where the whole set of triples that can be built from the sample set is not considered. We just focus on a small part of the candidate triples. Namely, in order to restrict the scope of the search, we first look for examples that are as similar as possible to the new item to be classified. We then only consider the pairs of examples presenting the same dissimilarity as between the new item and one of its closest neighbors. In this way, we implicitly build triples that are in analogical proportion on all attributes with the new item. Then the classification is made on the basis of an additive aggregation of the truth values corresponding to the pairs that can be analogically associated with the pairs made of the target item and one of its nearest neighbors. We then only deal with pairs leading to a solvable analogical equation for the class. This new algorithm provides results as good as previous analogical classifiers with a lower average complexity, both in nominal and numerical cases.

MSC:

68T05 Learning and adaptive systems in artificial intelligence

Software:

UCI-ml; LIBSVM
Full Text: DOI

References:

[1] Bayoudh, S.; Miclet, L.; Delhay, A., Learning by analogy: a classification rule for binary and nominal data, (Proc. Int. Joint Conf. on Artificial Intelligence. Proc. Int. Joint Conf. on Artificial Intelligence, IJCAI07 (2007)), 678-683
[2] Bounhas, M.; Prade, H.; Richard, G., Analogical classification: a new way to deal with examples, (Proc. 21st Europ. Conf. on Artificial Intelligence. Proc. 21st Europ. Conf. on Artificial Intelligence, ECAI’14, Prague, Aug. 18-22. Proc. 21st Europ. Conf. on Artificial Intelligence. Proc. 21st Europ. Conf. on Artificial Intelligence, ECAI’14, Prague, Aug. 18-22, Frontiers in Artificial Intelligence and Applications, vol. 263 (2014), IOS Press), 135-140 · Zbl 1366.68229
[3] Bounhas, M.; Prade, H.; Richard, G., Analogical classification: a rule-based view, (Proc. 15th Int. Conf. on Information Processing and Management of Uncertainty in Knowledge-Based Systems, Part II. Proc. 15th Int. Conf. on Information Processing and Management of Uncertainty in Knowledge-Based Systems, Part II, IPMU’14, Montpellier, July 15-19 (2014)), 485-495
[4] Bounhas, M.; Prade, H.; Richard, G., Analogical classification: handling numerical data, (Straccia, U.; Calì, A., Proc. 8th Int. Conf. on Scalable Uncertainty Management. Proc. 8th Int. Conf. on Scalable Uncertainty Management, SUM’14. Proc. 8th Int. Conf. on Scalable Uncertainty Management. Proc. 8th Int. Conf. on Scalable Uncertainty Management, SUM’14, Lect. Notes Comput. Sci., vol. 8720 (2014), Springer), 66-79
[5] Bounhas, M.; Prade, H.; Richard, G., Evenness-based reasoning with logical proportions applied to classification, (Proc. 9th Int. Conf. on Scalable Uncertainty Management. Proc. 9th Int. Conf. on Scalable Uncertainty Management, SUM’15. Proc. 9th Int. Conf. on Scalable Uncertainty Management. Proc. 9th Int. Conf. on Scalable Uncertainty Management, SUM’15, Lect. Notes Comput. Sci., vol. 9310 (2015), Springer), 139-154
[6] Bounhas, M.; Prade, H.; Richard, G., A new view of conformity and its application to classification, (van der Gaag, L. C., Proc. 13th Eur. Conf. on Symb. and Quantit. Appr. to Reasoning with Uncertainty. Proc. 13th Eur. Conf. on Symb. and Quantit. Appr. to Reasoning with Uncertainty, ECSQARU’15, Compigne, France, Jul. 15-17. Proc. 13th Eur. Conf. on Symb. and Quantit. Appr. to Reasoning with Uncertainty. Proc. 13th Eur. Conf. on Symb. and Quantit. Appr. to Reasoning with Uncertainty, ECSQARU’15, Compigne, France, Jul. 15-17, Lect. Notes Comput. Sci. (2015), Springer) · Zbl 1465.62118
[7] Bounhas, M.; Prade, H.; Richard, G., Oddness-based classifiers for boolean or numerical data, (Proc. 38th Annual German Conference on AI, Advances in Artificial Intelligence. Proc. 38th Annual German Conference on AI, Advances in Artificial Intelligence, KI’15, Dresden, Sept. 21-25. Proc. 38th Annual German Conference on AI, Advances in Artificial Intelligence. Proc. 38th Annual German Conference on AI, Advances in Artificial Intelligence, KI’15, Dresden, Sept. 21-25, Lect. Notes Comput. Sci., vol. 9324 (2015), Springer), 32-44
[8] Bounhas, M.; Prade, H.; Richard, G., Oddness/evenness-based classifiers for boolean or numerical data, Int. J. Approx. Reas., 82, 81-100 (2017) · Zbl 1410.62101
[9] Cheng, W.w.; Hüllermeier, E., Combining instance-based learning and logistic regression for multilabel classification, Mach. Learn., 76, 2-3, 211-225 (Sep. 2009) · Zbl 1470.68091
[10] Conover, W. J., Practical Nonparametric Statistics (1971), John Wiley: John Wiley New York · Zbl 0151.23503
[11] Correa, W.; Prade, H.; Richard, G., Trying to understand how analogical classifiers work, (Hüllermeier, E.; etal., Proc. 6th Int. Conf. on Scalable Uncertainty Management. Proc. 6th Int. Conf. on Scalable Uncertainty Management, SUM’12, Marburg, Germany. Proc. 6th Int. Conf. on Scalable Uncertainty Management. Proc. 6th Int. Conf. on Scalable Uncertainty Management, SUM’12, Marburg, Germany, Lect. Notes Comput. Sci., vol. 7520 (2012), Springer Verlag), 582-589
[12] Correa, W.; Prade, H.; Richard, G., When intelligence is just a matter of copying, (Proc. 20th Eur. Conf. on Artificial Intelligence. Proc. 20th Eur. Conf. on Artificial Intelligence, Montpellier, Aug. 27-31 (2012), IOS Press), 276-281 · Zbl 1327.68306
[13] Couceiro, M.; Hug, N.; Prade, H.; Richard, G., Analogy-preserving functions: a way to extend Boolean samples, (Proc. of 26th Int. Joint Conf. on Artificial Intelligence. Proc. of 26th Int. Joint Conf. on Artificial Intelligence, IJCAI17, Melbourne (2017)), 1575-1581
[14] Davies, T. R.; Russell, S. J., A logical approach to reasoning by analogy, (McDermott, J. P., Proc. of the 10th International Joint Conference on Artificial Intelligence. Proc. of the 10th International Joint Conference on Artificial Intelligence, IJCAI’87, Milan (1987), Morgan Kaufmann), 264-270
[15] Dubois, D.; Prade, H.; Richard, G., Multiple-valued extensions of analogical proportions, Fuzzy Sets Syst., 292, 193-202 (2016) · Zbl 1380.03029
[16] Friedman, M., The use of ranks to avoid the assumption of normality implicit in the analysis of variance, J. Am. Stat. Assoc., 32, 200, 675-701 (1937) · JFM 63.1098.02
[17] Gergonne, J. D., Application de la méthode des moindres quarrés à l’interpolation des suites, Ann. Math. Pures Appl., 6, 242-252 (1815)
[18] Gergonne, J. D., Théorie de la règle de trois, Ann. Math. Pures Appl., 7, 117-122 (1816)
[19] Hesse, M., On defining analogy, Proc. Aristot. Soc., 60, 79-100 (1959)
[20] Hug, N.; Prade, H.; Richard, G.; Serrurier, M., Analogical classifiers: a theoretical perspective, (Proc. 22nd Europ. Conf. on Artificial Intelligence. Proc. 22nd Europ. Conf. on Artificial Intelligence, ECAI’16, The Hague. Proc. 22nd Europ. Conf. on Artificial Intelligence. Proc. 22nd Europ. Conf. on Artificial Intelligence, ECAI’16, The Hague, Frontiers in Artificial Intelligence and Applications (2016), IOS Press) · Zbl 1403.68191
[21] Jarmulak, J.; Craw, S.; Rowe, R., Using case-base data to learn adaptation knowledge for design, (Proceedings of the 17th International Joint Conference on Artificial Intelligence, vol. 2. Proceedings of the 17th International Joint Conference on Artificial Intelligence, vol. 2, IJCAI’01, San Francisco, CA, USA (2001), Morgan Kaufmann Publishers Inc.), 1011-1016
[22] Langley, P.; Iba, W., Average-case analysis of a nearest neighbor algorithm, (Proc. 13th Int. Joint Conf. on Artificial Intelligence. Proc. 13th Int. Joint Conf. on Artificial Intelligence, Chambéry, France, August 28 - September 3 (1993), Morgan Kaufmann), 889-894
[23] Lavallée, J. F.; Langlais, P., Moranapho: un système multilingue d’analyse morphologique basé sur l’analogie formelle, TAL, Trait. Autom. Lang., 52, 2, 17-44 (2011)
[24] Lepage, Y., Analogy and formal languages, Electron. Notes Theor. Comput. Sci., 53 (2001) · Zbl 1263.68100
[25] McSherry, D., Case-based reasoning techniques for estimation, (IEE Colloquium on Case-Based Reasoning (Feb. 1993)), 6/1-6/4
[26] Melis, E.; Veloso, M., Analogy in problem solving, (Handbook of Practical Reasoning: Comput. and Theor. Aspects (1998), OUP)
[27] Mertz, J.; Murphy, P. M., Uci repository of machine learning databases (2000), Available at
[28] Miclet, L.; Barbot, N.; Jeudy, B., Analogical proportions in a lattice of sets of alignments built on the common subwords in a finite language, (Prade, H.; Richard, G., Computational Approaches to Analogical Reasoning - Current Trends (2013), Springer), 245-260 · Zbl 1341.68145
[29] Miclet, L.; Barbot, N.; Prade, H., From analogical proportions in lattices to proportional analogies in formal concepts, (Proc. 21st Europ. Conf. on Artificial Intelligence. Proc. 21st Europ. Conf. on Artificial Intelligence, ECAI’14, Prague, 18-22 Aug.. Proc. 21st Europ. Conf. on Artificial Intelligence. Proc. 21st Europ. Conf. on Artificial Intelligence, ECAI’14, Prague, 18-22 Aug., Frontiers in Artificial Intelligence and Applications, vol. 263 (2014), IOS Press), 627-632 · Zbl 1366.68310
[30] Miclet, L.; Bayoudh, S.; Delhay, A., Analogical dissimilarity: definition, algorithms and two experiments in machine learning, J. Artif. Intell. Res., 32, 793-824 (2008) · Zbl 1183.68489
[31] Miclet, L.; Delhay, A., Relation d’analogie et distance sur un alphabet défini par des traits (July 2004), IRISA, Technical Report 1632
[32] Miclet, L.; Prade, H., Handling analogical proportions in classical logic and fuzzy logics settings, (Proc. 10th Eur. Conf. on Symbolic and Quantitative Approaches to Reasoning with Uncertainty. Proc. 10th Eur. Conf. on Symbolic and Quantitative Approaches to Reasoning with Uncertainty, ECSQARU’09, Verona. Proc. 10th Eur. Conf. on Symbolic and Quantitative Approaches to Reasoning with Uncertainty. Proc. 10th Eur. Conf. on Symbolic and Quantitative Approaches to Reasoning with Uncertainty, ECSQARU’09, Verona, Lect. Notes Comput. Sci., vol. 5590 (2009), Springer), 638-650 · Zbl 1245.03046
[33] Moraes, R. M.; Machado, L. S.; Prade, H.; Richard, G., Classification based on homogeneous logical proportions, (Bramer, M.; Petridis, M., Proc. of AI-2013, The Thirty-third SGAI International Conference on Innovative Techniques and Applications of Artificial Intelligence. Proc. of AI-2013, The Thirty-third SGAI International Conference on Innovative Techniques and Applications of Artificial Intelligence, Cambridge, England, UK (2013), Springer), 53-60
[34] Okamoto, S.; Yugami, N., Theoretical analysis of the nearest neighbor classifier in noisy domains, (Proc. 13th International Conference on Machine Learning (1996), Morgan Kaufmann), 355-363
[35] Prade, H.; Richard, G., Reasoning with logical proportions, (Lin, F. Z.; Sattler, U.; Truszczynski, M., Proc. 12th Int. Conf. on Principles of Knowledge Representation and Reasoning. Proc. 12th Int. Conf. on Principles of Knowledge Representation and Reasoning, KR 2010, Toronto, May 9-13, 2010 (2010), AAAI Press), 545-555
[36] Prade, H.; Richard, G., Homogeneous logical proportions: their uniqueness and their role in similarity-based prediction, (Brewka, G.; Eiter, T.; McIlraith, S. A., Proc. 13th Int. Conf. on Principles of Knowledge Representation and Reasoning. Proc. 13th Int. Conf. on Principles of Knowledge Representation and Reasoning, KR’12, Roma, June 10-14 (2012), AAAI Press), 402-412
[37] Prade, H.; Richard, G., Analogical proportions and multiple-valued logics, (van der Gaag, L. C., Proc. 12th Eur. Conf. on Symb. and Quantit. Appr. to Reasoning with Uncertainty. Proc. 12th Eur. Conf. on Symb. and Quantit. Appr. to Reasoning with Uncertainty, ECSQARU’13, Utrecht, Jul. 7-10. Proc. 12th Eur. Conf. on Symb. and Quantit. Appr. to Reasoning with Uncertainty. Proc. 12th Eur. Conf. on Symb. and Quantit. Appr. to Reasoning with Uncertainty, ECSQARU’13, Utrecht, Jul. 7-10, Lect. Notes Comput. Sci., vol. 7958 (2013), Springer), 497-509 · Zbl 1390.68615
[38] Prade, H.; Richard, G., From analogical proportion to logical proportions, Log. Univers., 7, 4, 441-505 (2013) · Zbl 1323.03011
[39] Prade, H.; Richard, G., Homogenous and heterogeneous logical proportions, IfCoLog J. Log. Appl., 1, 1, 1-51 (2014)
[40] Prade, H.; Richard, G., Multiple-valued logic interpretations of analogical, reverse analogical, and paralogical proportions, (Proc. 40th IEEE Int. Symp. on Multiple-Valued Logic. Proc. 40th IEEE Int. Symp. on Multiple-Valued Logic, ISMVL’10, Barcelona (2010)), 258-263
[41] Prade, H.; Richard, G.; Yao, B., Enforcing regularity by means of analogy-related proportions-a new approach to classification, Int. J. Comput. Inf. Syst. Ind. Manag. Appl., 4, 648-658 (2012)
[42] Ragni, M.; Neubert, S., Analyzing raven’s intelligence test: cognitive model, demand, and complexity, (Computational Approaches to Analogical Reasoning: Current Trends. Computational Approaches to Analogical Reasoning: Current Trends, Studies in Computational Intelligence, vol. 548 (2014), Springer), 351-370
[43] Rumelhart, D. E.; Abrahamson, A. A., A model for analogical reasoning, Cogn. Psychol., 5, 1-28 (2005)
[44] Stroppa, N.; Yvon, F., An analogical learner for morphological analysis, (Online Proc. 9th Conf. Comput. Natural Language Learning. Online Proc. 9th Conf. Comput. Natural Language Learning, CoNLL-2005 (2005)), 120-127
[45] Stroppa, N.; Yvon, F., Analogical Learning and Formal Proportions: Definitions and Methodological Issues (June 2005), ENST, Technical report
[46] Stroppa, N.; Yvon, F., Du quatrième de proportion comme principe inductif : une proposition et son application à l’apprentissage de la morphologie, TAL, Trait. Autom. Lang., 47, 2, 1-27 (2006)
[48] Yvon, F.; Stroppa, N., Formal Models of Analogical Proportions (2006), Technical report, Ecole Nationale Supérieure des Télécommunications, no D008
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.