Using robust dispersion estimation in support vector machines. (English) Zbl 1326.68234
Summary: In this paper, a novel Support Vector Machine (SVM) variant, which makes use of robust statistics, is proposed. We investigate the use of statistically robust location and dispersion estimators, in order to enhance the performance of SVMs and test it in two-class and multi-class classification problems. Moreover, we propose a novel method for class specific multi-class SVM, which makes use of the covariance matrix of only one class, i.e., the class that we are interested in separating from the others, while ignoring the dispersion of other classes. We performed experiments in artificial data, as well as in many real world publicly available databases used for classification. The proposed approach performs better than other SVM variants, especially in cases where the training data contain outliers. Finally, we applied the proposed method for facial expression recognition in three well known facial expression databases, showing that it outperforms previously published attempts.
MSC:
| 68T05 | Learning and adaptive systems in artificial intelligence |
References:
| [1] | Shivaswamy, P.; Jebara, T., Maximum relative margin and data-dependent regularization, The Journal of Machine Learning Research, 11, 747-788 (2010) · Zbl 1242.68249 |
| [2] | Adankon, M.; Cheriet, M., Model selection for the LS-SVM application to handwriting recognition, Pattern Recognition, 42, 12, 3264-3270 (2009) · Zbl 1187.68424 |
| [3] | Khan, N.; Ksantini, R.; Ahmad, I.; Boufama, B., A novel SVM+NDA model for classification with an application to face recognition, Pattern Recognition, 45, 1, 66-79 (2012) · Zbl 1225.68230 |
| [4] | Wang, X.; SHU-XIA, L.; Zhai, J., Fast fuzzy multicategory SVM based on support vector domain description, International Journal of Pattern Recognition and Artificial Intelligence, 22, 01, 109-120 (2008) |
| [5] | Wu, Y.; Lee, Y.; Yang, J., Robust and efficient multiclass SVM models for phrase pattern recognition, Pattern Recognition, 41, 9, 2874-2889 (2008) · Zbl 1167.68439 |
| [6] | Liu, R.; Wang, Y.; Baba, T.; Masumoto, D.; Nagata, S., SVM-based active feedback in image retrieval using clustering and unlabeled data, Pattern Recognition, 41, 8, 2645-2655 (2008) · Zbl 1151.68397 |
| [7] | Xue, Z.; Ming, D.; Song, W.; Wan, B.; Jin, S., Infrared gait recognition based on wavelet transform and support vector machine, Pattern Recognition, 43, 8, 2904-2910 (2010) · Zbl 1207.68330 |
| [8] | Tahir, M.; Khan, A.; Majid, A., Protein subcellular localization of fluorescence imagery using spatial and transform domain features, Bioinformatics, 28, 1, 91-97 (2012) |
| [9] | Khan, A.; Tahir, S. F.; Majid, A.; Choi, T.-S., Machine learning based adaptive watermark decoding in view of anticipated attack, Pattern Recognition, 41, 8, 2594-2610 (2008) · Zbl 1151.68585 |
| [10] | Wang, X.; Wang, T.; Bu, J., Color image segmentation using pixel wise support vector machine classification, Pattern Recognition, 44, 4, 777-787 (2011) · Zbl 1213.94023 |
| [11] | Williamson, R.; Smola, A.; Scholkopf, B., Generalization performance of regularization networks and support vector machines via entropy numbers of compact operators, IEEE Transactions on Information Theory, 47, 6, 2516-2532 (2002) · Zbl 1008.62507 |
| [12] | Huang, K.; Yang, H.; King, I.; Lyu, M., Maxi-Min margin machinelearning large margin classifiers locally and globally, IEEE Transactions on Neural Networks, 19, 2, 260-272 (2008) |
| [13] | Tefas, A.; Kotropoulos, C.; Pitas, I., Using support vector machines to enhance the performance of elastic graph matching for frontal face authentication, IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, 7, 735-746 (2002) |
| [14] | Zafeiriou, S.; Tefas, A.; Pitas, I., Minimum class variance support vector machines, IEEE Transactions on Image Processing, 16, 10, 2551-2564 (2007) |
| [15] | Kotsia, I.; Pitas, I., Facial expression recognition in image sequences using geometric deformation features and support vector machines, IEEE Transactions on Image Processing, 16, 1, 172-187 (2006) |
| [18] | Wilcox, R., Introduction to Robust Estimation and Hypothesis Testing (2005), Academic Press · Zbl 1113.62036 |
| [19] | Azzalini, A.; Genton, M., Robust likelihood methods based on the skew-t and related distributions, International Statistical Review, 76, 1, 106-129 (2008) · Zbl 1206.62102 |
| [20] | Hubert, M.; Debruyne, M., Minimum covariance determinant, Wiley Interdisciplinary ReviewsComputational Statistics, 2, 1, 36-43 (2010) |
| [21] | Hubert, M.; Rousseeuw, P.; Vanden Branden, K., ROBPCAa new approach to robust principal component analysis, Technometrics, 47, 1, 64-79 (2005) |
| [22] | Rousseeuw, P.; Leroy, A., Robust Regression and Outlier Detection (1987), John Wiley & Sons Inc · Zbl 0711.62030 |
| [23] | Croux, C.; Haesbroeck, G., Influence function and efficiency of the minimum covariance determinant scatter matrix estimator, Journal of Multivariate Analysis, 71, 2, 161-190 (1999) · Zbl 0946.62055 |
| [24] | Rousseeuw, P.; Van Driessen, K., A fast algorithm for the minimum covariance determinant estimator, Technometrics, 41, 3, 212-223 (1999) |
| [25] | Schyns, M.; Haesbroeck, G.; Critchley, F., Relaxmcdsmooth optimisation for the minimum covariance determinant estimator, Computational Statistics & Data Analysis, 54, 4, 843-857 (2010) · Zbl 1464.62156 |
| [26] | Bernholt, T.; Fischer, P., The complexity of computing the MCD-estimator, Theoretical Computer Science, 326, 1-3, 383-398 (2004) · Zbl 1054.62028 |
| [27] | Yang, F.; Shan, Z.; Kruggel, F., White matter lesion segmentation based on feature joint occurrence probability and \(\chi 2\) random field theory from magnetic resonance (mr) images, Pattern Recognition Letters, 31, 9, 781-790 (2010) |
| [28] | Zhan, Y.; Yin, J., Robust local tangent space alignment via iterative weighted PCA, Neurocomputing, 74, 11, 1985-1993 (2011) |
| [29] | Jin, J.; An, J., Robust discriminant analysis and its application to identify protein coding regions of rice genes, Mathematical Biosciences, 232, 2, 96-100 (2011) · Zbl 1219.62100 |
| [30] | Vapnik, V., The Nature of Statistical Learning Theory (2000), Springer Verlag · Zbl 0934.62009 |
| [31] | McLachlan, G.; Wiley, J., Discriminant Analysis and Statistical Pattern Recognition (1992), Wiley Online Library · Zbl 0850.62481 |
| [32] | Fletcher, R., Practical Methods of Optimization (1987), John Wiley & Sons Inc · Zbl 0905.65002 |
| [33] | Girolami, M., Mercer kernel-based clustering in feature space, IEEE Transactions on Neural Networks, 13, 3, 780-784 (2002) |
| [34] | Hubert, M.; Rousseeuw, P.; Verboven, S., A fast method for robust principal components with applications to chemometrics, Chemometrics and Intelligent Laboratory Systems, 60, 1-2, 101-111 (2002) |
| [35] | Hsu, C.; Lin, C., A comparison of methods for multiclass support vector machines, IEEE Transactions on Neural Networks, 13, 2, 415-425 (2002) |
| [42] | Lindgren, B., Statistical Theory (1993), Chapman & Hall/CRC · Zbl 0188.49301 |
| [43] | Yang, J.; Frangi, A.; Yang, J.; Zhang, D.; Jin, Z., KPCA plus LDAa complete kernel Fisher discriminant framework for feature extraction and recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 230-244 (2005) |
| [45] | Draper, B.; Baek, K.; Bartlett, M.; Beveridge, J., Recognizing faces with PCA and ICA, Computer Vision and Image Understanding, 91, 1-2, 115-137 (2003) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
