Abstract
Support Vector Machine (SVM) results in a good generalization performance by employing the Structural Risk Minimization (SRM) principle. However, one drawback is O(n 3) training time complexity. In this paper, we propose a pattern selection method designed specifically for Support Vector Regression (SVR). In SVR training, only a few patterns called support vectors are used to construct the regression model while other patterns are not used at all. The proposed method tries to select patterns which are likely to become support vectors. With multiple bootstrap samples, we estimate the likelihood of each pattern to become a support vector. The proposed method automatically determines the appropriate number of patterns selected by estimating the expected number of support vectors. Through the experiments involving twenty datasets, the proposed method resulted in the best accuracy among the competing methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge (2000)
Drucker, H., Burges, C.J.C., Kaufman, L., Smola, A., Vapnik, V.: Support Vector Regression Machines. In: Mozer, M.C., Jordan, M.I., Petsche, T. (eds.) Advances in Neural Information Processing System, vol. 9. MIT Press, Cambridge (1997)
Platt, J.C.: Fast Training of Support Vector Machines Using Sequential Minimal Optimization. In: Advanced in Kernel Methods; Support Vector Machines, pp. 185–208. MIT Press, Cambridge (1999)
Almeida, M.B., Braga, A., Braga, J.P.: SVM-KM: Speeding SVMs Learning with a Priori Cluster Selection and k-Means. In: Proc. of the 6th Brazilian Symposium on Neural Networks, pp. 162–167 (2000)
Shin, H., Cho, S.: Neighborhood Property based Pattern Selection for SVM. Neural Computation 19(3), 816–855 (2007)
Bakir, G.H., Bottou, L., Weston, J.: Breaking SVM Complexity with Cross-Training. In: Advances in Neural Information Processing Systems, vol. 17, pp. 81–88 (2005)
Joachims, T.: Training Linear SVMs in Linear Time. In: Proc. of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 217–226 (2006)
Wang, W., Xu, Z.: A Heuristic Training for Support Vector Regression. Neurocomputing 61, 259–275 (2004)
Sun, J., Cho, S.: Pattern Selection for Support Vector Regression based on Sparsity and Variability. In: 2006 IEEE International Joint Conference on Neural Networks (IJCNN), pp. 559–602 (2006)
Kim, D., Cho, S.: ε–tube based Pattern Selection for Support Vector Machines. In: Ng, W.-K., Kitsuregawa, M., Li, J., Chang, K. (eds.) PAKDD 2006. LNCS (LNAI), vol. 3918, pp. 215–224. Springer, Heidelberg (2006)
Kim, D., Lee, H., Cho, S.: Response Modeling with Support Vector Regression. Expert Systems with Applications 34(2), 1102–1108 (2008)
Chalimourda, A., Schölkopf, B., Smola, A.: Experimentally Optimal ν in Support Vector Regression for Different Noise Models and Parameter Settings. Neural Networks 17, 127–141 (2004)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kim, D., Cho, S. (2008). Bootstrap Based Pattern Selection for Support Vector Regression. In: Washio, T., Suzuki, E., Ting, K.M., Inokuchi, A. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2008. Lecture Notes in Computer Science(), vol 5012. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-68125-0_56
Download citation
DOI: https://doi.org/10.1007/978-3-540-68125-0_56
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-68124-3
Online ISBN: 978-3-540-68125-0
eBook Packages: Computer ScienceComputer Science (R0)