A coordinate descent margin based-twin support vector machine for classification. (English) Zbl 1259.68173
Summary: Twin support vector machines (TWSVMs) obtain faster learning speed by solving a pair of smaller SVM-type problems. In order to increase its efficiency further, this paper presents a coordinate descent margin based twin vector machine (CDMTSVM) compared with the original TWSVM. The major advantages of CDMTSVM lie in two aspects: (1) The primal and dual problems are reformulated and improved by adding a regularization term in the primal problems which implies maximizing the “margin” between the proximal hyperplane and bounding hyperplane, yielding the dual problems to be stable positive definite quadratic programming problems. (2) A novel coordinate descent method is proposed for our dual problems which leads to very fast training. As our coordinate descent method handles one data point at a time, it can process very large datasets that need not reside in memory. Our experiments on publicly available datasets indicate that our CDMTSVM is not only fast, but also shows good generalization performance.
MSC:
| 68T05 | Learning and adaptive systems in artificial intelligence |
Keywords:
pattern classification; twin support vector machines; maximum margin; coordinate descent methodReferences:
| [1] | Blake, C. L., & Merz, C. J. (1998). UCI repository for machine learning databases. http://www.ics.uci.edu/ mlearn/MLRepository.html; Blake, C. L., & Merz, C. J. (1998). UCI repository for machine learning databases. http://www.ics.uci.edu/ mlearn/MLRepository.html |
| [2] | Burges, C., A tutorial on support vector machines for pattern recognition, Data Mining and Knowledge Discovery, 2, 1-43 (1998) |
| [3] | Chang, K. W.; Hsieh, C. J.; Lin, C. J., Coordinate descent method for large-scale L2-loss linear SVM, Journal of Machine Learning Research, 9, 1369-1398 (2008) · Zbl 1225.68157 |
| [4] | Chang, C. C., & Lin, C. J. (2001). LIBSVM: a library for support vector machines. http://www.csie.ntu.edu.tw/ cjlin; Chang, C. C., & Lin, C. J. (2001). LIBSVM: a library for support vector machines. http://www.csie.ntu.edu.tw/ cjlin |
| [5] | Cortes, C.; Vapnik, V. N., Support vector networks, Machine Learning, 20, 273-297 (1995) · Zbl 0831.68098 |
| [6] | Duda, R. O.; Hart, P. E.; Stork, D. G., Pattern classification (2001), John Wiley and Sons · Zbl 0968.68140 |
| [7] | Fan, R. E.; Chang, K. W.; Hsieh, C. J.; Wang, X. R.; Lin, C. J., LIBLINEAR: a library for large linear classification, Journal of Machine Learning Research, 9, 1871-1874 (2008) · Zbl 1225.68175 |
| [8] | Fung, G., & Mangasarian, O. L. (2001). Proximal support vector machine classifiers. In Proceedings of seventh international conference on knowledge and data discovery; Fung, G., & Mangasarian, O. L. (2001). Proximal support vector machine classifiers. In Proceedings of seventh international conference on knowledge and data discovery · Zbl 1101.68758 |
| [9] | Golub, G. H.; Van Loan, C. F., Matrix computations (1996), The JohnHopkins University Press, Chapter 2, p. 50 · Zbl 0865.65009 |
| [10] | Gunn, S. R. (1998). Support vector machines for classification and regression. http://www.isis.ecs.soton.ac.uk/resources/svminfo/; Gunn, S. R. (1998). Support vector machines for classification and regression. http://www.isis.ecs.soton.ac.uk/resources/svminfo/ |
| [11] | Hsieh, C.-J., Chang, K.-W., & Lin, C.-J. (2008). A dual coordinate descent method for large-scale linear SVM. In International conference on machine learning; Hsieh, C.-J., Chang, K.-W., & Lin, C.-J. (2008). A dual coordinate descent method for large-scale linear SVM. In International conference on machine learning |
| [12] | Ince, H., & Trafalis, T. B. (2002). Support vector machine for regression and applications to financial forecasting. In International joint conference on neural networks; Ince, H., & Trafalis, T. B. (2002). Support vector machine for regression and applications to financial forecasting. In International joint conference on neural networks |
| [13] | Jayadeva; Khemchandani, R.; Chandra, S., Twin support vector machines for pattern classification, IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 905-910 (2007) |
| [14] | Joachims, T. (1998). Making large-scale SVM learning practical. In Advances in kernel methods-support vector learning; Joachims, T. (1998). Making large-scale SVM learning practical. In Advances in kernel methods-support vector learning |
| [15] | Khemchandani, R.; Jayadeva; Chandra, S., Optimal kernel selection in twin support vector machines, Optimization Letters, 3, 77-88 (2009) · Zbl 1154.90567 |
| [16] | Kumar, M. A.; Gopal, M., Application of smoothing technique on twin support vector machines, Pattern Recognition Letters, 29, 1842-1848 (2008) |
| [17] | Lal, T. N.; Schröder, M.; Hinterberger, T.; Weston, J.; Bogdan, M.; Birbaumer, N., Support vector channel selection in BCI, Data Mining and Knowledge Discovery, 51, 1003-1010 (2004) |
| [18] | Lee, Y. J., & Mangasarian, O. L. (2001). RSVM: reduced support vector machines. In First SIAM international conference on data mining; Lee, Y. J., & Mangasarian, O. L. (2001). RSVM: reduced support vector machines. In First SIAM international conference on data mining |
| [19] | Luo, Z. Q.; Tseng, P., Error bounds and convergence analysis of feasible descent methods: a general approach, Annals of Operations Research, 46, 157-178 (1993) · Zbl 0793.90076 |
| [20] | Mangasarian, O. L., Nonlinear programming (1994), SIAM · Zbl 0194.20201 |
| [21] | Mangasarian, O. L.; Musicant, D. R., Successive overrelaxation for support vector machines, IEEE Transactions on Neural Networks, 10, 1032-1037 (1999) |
| [22] | Mangasarian, O. L.; Wild, E. W., Multisurface proximal support vector classification via generalize deigenvalues, IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 69-74 (2006) |
| [23] | MATLAB. 2007. The MathWorks, Inc. http://www.mathworks.com; MATLAB. 2007. The MathWorks, Inc. http://www.mathworks.com |
| [24] | Musicant, D. R. (1998). NDC: normally distributed clustered datasets. http://www.cs.wisc.edu/dmi/svm/ndc/; Musicant, D. R. (1998). NDC: normally distributed clustered datasets. http://www.cs.wisc.edu/dmi/svm/ndc/ |
| [25] | Noble, W. S. (2004). Support vector machine applications in computational biology. In Kernel methods in computational biology; Noble, W. S. (2004). Support vector machine applications in computational biology. In Kernel methods in computational biology |
| [26] | Platt, J., Fast training of support vector machines using sequential minimal optimization, (Advances in kernel methods-support vector learning (1999), MIT Press: MIT Press Cambridge, MA), 185-2008 |
| [27] | Scholköpf, B.; Smola, A., Learning with kernels (2002), MIT Press: MIT Press Cambridge, MA |
| [28] | Suykens, J. A. K., Lukas, L., Van Dooren, P., De Moor, B., & Vandewalle, J. (1999). Least squares support vector machine classifiers: a large scale algorithm. In Proceedings of European conference of circuit theory design; Suykens, J. A. K., Lukas, L., Van Dooren, P., De Moor, B., & Vandewalle, J. (1999). Least squares support vector machine classifiers: a large scale algorithm. In Proceedings of European conference of circuit theory design |
| [29] | Suykens, J. A.K.; Vandewalle, J., Least squares support vector machine classifiers, Neural Processing Letters, 9, 293-300 (1999) · Zbl 0958.93042 |
| [30] | Zhang, C. H.; Tian, Y. J.; Deng, N. Y., The new interpretation of support vector machines on statistical learning theory, Science China, 53, 151-164 (2010) · Zbl 1187.90221 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
