Combining multi-class SVMs with linear ensemble methods that estimate the class posterior probabilities. (English) Zbl 1290.68109
Summary: Roughly speaking, there is one main model of pattern recognition support vector machine, with several variants of lower popularity. On the contrary, among the different multi-class support vector machines which can be found in the literature, none is clearly favoured. On the one hand, they exhibit distinct statistical properties. On the other hand, multiple comparative studies between multi-class support vector machines and decomposition methods have highlighted the fact that each model has its advantages and drawbacks. These observations call for the evaluation of combinations of multi-class support vector machines. In this article, we study the combination of multi-class support vector machines with linear ensemble methods. Their sample complexity is low, which should prevent them from overfitting, and the outputs of two of them are estimates of the class posterior probabilities.
MSC:
| 68T10 | Pattern recognition, speech recognition |
| 62G08 | Nonparametric regression and quantile regression |
| 68T05 | Learning and adaptive systems in artificial intelligence |
Keywords:
capacity control; class posterior probability estimates; ensemble methods; multi-class support vector machines; multivariate linear modelsReferences:
| [1] | DOI: 10.1007/978-1-4419-9096-9 · doi:10.1007/978-1-4419-9096-9 |
| [2] | Bonidal R., SMTDA’12 (2012) |
| [3] | Boser , B. E. , Guyon , I. M. , Vapnik , V. N. ( 1992 ). A training algorithm for optimal margin classifiers .COLT’92, Pittsburgh , PA , pp. 144 – 152 . |
| [4] | DOI: 10.1111/1467-9868.00054 · doi:10.1111/1467-9868.00054 |
| [5] | DOI: 10.1023/A:1022627411411 · doi:10.1023/A:1022627411411 |
| [6] | DOI: 10.1162/15324430260185628 · Zbl 1037.68110 · doi:10.1162/15324430260185628 |
| [7] | Guermeur Y., Patt. Anal. Applic. 5 (2) pp 168– (2002) · Zbl 1021.68080 · doi:10.1007/s100440200015 |
| [8] | Guermeur , Y. ( 2007 ).SVM Multiclasses, Théorie et Applications.Habilitation à diriger des recherches, Thesis, UHP . |
| [9] | Guermeur Y., Commun. Statist. Theor. Meth. 39 (3) pp 543– (2010) · Zbl 1188.62188 · doi:10.1080/03610920903140288 |
| [10] | Guermeur Y., Int. J. Intell. Inform. Database Syst. 6 (6) pp 555– (2012) |
| [11] | Guermeur Y., Informatica 22 (1) pp 73– (2011) |
| [12] | DOI: 10.1016/0893-6080(89)90020-8 · Zbl 1383.92015 · doi:10.1016/0893-6080(89)90020-8 |
| [13] | Hosmer D. W., Applied Logistic Regression (1989) · Zbl 0967.62045 |
| [14] | DOI: 10.1109/72.991427 · doi:10.1109/72.991427 |
| [15] | Krogh A., Neur. Computat. 11 (2) pp 541– (1999) · doi:10.1162/089976699300016764 |
| [16] | DOI: 10.1198/016214504000000098 · Zbl 1089.62511 · doi:10.1198/016214504000000098 |
| [17] | Lin H.-T., Mach. Learn. 68 (3) pp 267– (2007) · Zbl 1471.68220 · doi:10.1007/s10994-007-5018-6 |
| [18] | Liu Y., Eleven Int. Conf. Artific. Intell. Statist. (2007) |
| [19] | Natarajan B., Mach. Learn. 4 (1) pp 67– (1989) |
| [20] | Platt J. C., Advances in Large Margin Classifiers pp 61– (2000) |
| [21] | Richard M. D., Neural Computat. 3 (4) pp 461– (1991) · doi:10.1162/neco.1991.3.4.461 |
| [22] | Rosen J. B., J. Soc. Indust. Appl. Math. 8 (1) pp 181– (1960) · Zbl 0099.36405 · doi:10.1137/0108011 |
| [23] | Schölkopf , B. , Burges , C. , Vapnik , V. ( 1995 ). Extracting support data for a given task .KDD’95, Menlo Park , CA , pp. 252 – 257 . |
| [24] | Suykens J. A. K., Neur. Process. Lett. 9 (3) pp 293– (1999) · doi:10.1023/A:1018628609742 |
| [25] | Tewari A., J. Mach. Learn. Res. 8 pp 1007– (2007) |
| [26] | Thomarat , F. , Lauer , F. , Guermeur , Y. ( 2012 ). Cascading discriminant and generative models for protein secondary structure prediction .PRIB’12, Tokyo , Japan , pp. 166 – 177 . |
| [27] | Vapnik V. N., Estimation of Dependences Based on Empirical Data (1982) · Zbl 0499.62005 |
| [28] | Vapnik V. N., Statistical Learning Theory (1998) · Zbl 0935.62007 |
| [29] | Wahba G., Nonlinear Modeling and Forecasting, SFI Studies in the Sciences of Complexity pp 95– (1992) |
| [30] | Weston J., Technical Report CSD-TR-98-04 (1998) |
| [31] | Zhang , Z. , Jordan , M. I. ( 2006 ). Bayesian multicategory support vector machines .UAI’06, Arlington , VA , pp. 552 – 559 . |
| [32] | Zou , H. , Zhu , J. , Hastie , T. ( 2006 ). The margin vector, admissible loss and multi-class margin-based classifiers. Technical Report, School of Statistics, University of Minnesota . |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
