Experimentally optimal \(\nu\) in support vector regression for different noise models and parameter settings. (English) Zbl 1072.68541
Summary: In Support Vector (SV) regression, a parameter \(\nu\) controls the number of Support Vectors and the number of points that come to lie outside of the so-called \(\varepsilon\)-insensitive tube. For various noise models and SV parameter settings, we experimentally determine the values of \(\nu\) that lead to the lowest generalization error. We find good agreement with the values that had previously been predicted by a theoretical argument based on the asymptotic efficiency of a simplified model of SV regression. As a side effect of the experiments, valuable information about the generalization behavior of the remaining SVM parameters and their dependencies is gained. The experimental findings are valid even for complex ‘real-world’ data sets. Based on our results on the role of the \(\nu\)-SVM parameters, we discuss various model selection methods.
MSC:
| 68T05 | Learning and adaptive systems in artificial intelligence |
Keywords:
Support Vector machines; \(\nu\)-Support Vector machines; Support Vector regression; Support Vector machine parameters; Optimal \(\nu\); Gaussian kernel; Model selection; Risk minimizationReferences:
| [1] | Cherkassky, V.; Ma, Y., Selection of meta-parameters for support vector regression. Selection of meta-parameters for support vector regression, In Proceedings of the International Conference on Artificial Neural Networks (ICANN), Madrid Spain (2002) · Zbl 1013.68844 |
| [2] | Cristianini, N.; Shawe-Taylor, J., An introduction to support vector machines (2000), Cambridge University Press |
| [3] | Duan, K., Keerthi, S. & Poo, A (2001). Evaluation of simple performance measures for tuning SVM hyperparameters; Duan, K., Keerthi, S. & Poo, A (2001). Evaluation of simple performance measures for tuning SVM hyperparameters |
| [4] | Mattera, D.; Haykin, S., Support vector machines for dynamic reconstruction of a chaotic system, (Schölkopf, B.; Burges, C.; Smola, A., Advances in kernel methods—Support vector learning (1999), MIT Press), 211-241 |
| [5] | Murata, N.; Yoshizawa, S.; Amari, S., Network information criterion—Determining the number of hidden units for artificial neural networks, IEEE Transactions on Neural Networks, 5, 865-872 (1994) |
| [6] | Schölkopf, B.; Smola, A. J.; Williamson, R. C.; Bartlett, P. L., New support vector algorithms, Neutral Computation, 12, 4, 1207-1245 (2000) |
| [7] | Smola, A. J.; Murata, N.; Schölkopf, B.; Müller, K.-R, Asymptotically optimal choice of \(ε\)-loss for support vector machines, (Niklasson, L.; Bodèn, M.; Ziemke, T., Perspectives in neural computing. Perspectives in neural computing, Proceedings of the Eighth International Conference on Artificial Neural Networks (1998), Springer Verlag), 105-110 |
| [8] | Stitson, M.; Gammerman, A.; Vapnik, V.; Vovk, V.; Watkins, C.; Weston, J., Support vector regression with ANOVA decomposition kernels, (Schölkopf, B.; Burges, C.; Smola, A., Advances in kernel methods—Support vector learning (1999), MIT Press), 285-291 |
| [9] | Vanderbei, R. J (1994). An interior point code for quadratic programming; Vanderbei, R. J (1994). An interior point code for quadratic programming · Zbl 0973.90518 |
| [10] | Vapnik, V., The nature of statistical learning theory (1995), Springer: Springer New York · Zbl 0833.62008 |
| [11] | Vapnik, V., Statistical learning theory (1998), Wiley: Wiley New York · Zbl 0935.62007 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
