Summary: The standard support vector machine (SVM) minimizes the hinge loss function subject to the \(L_{2}\) penalty or the roughness penalty. Recently, the \(L_{1}\) SVM was suggested for variable selection by producing sparse solutions [P. S. Bradley and O. L. Mangasarian, “Feature selection via concave minimization and support vector machines”, in: Proceedings of the Fifteenth International Conference on Machine Learning, ICML’98. San Francisco, CA: Morgan Kaufmann (1998; doi:10.5555/645527.657467); J. Zhu et al., “1-norm support vector machines”, in: Proceedings of the 16th international conference on neural information processing systems, NIPS’03. Cambridge, MA: MIT Press. 49–56 (2003; doi:10.5555/2981345.2981352)]. These learning methods are non-adaptive since their penalty forms are pre-determined before looking at data, and they often perform well only in a certain type of situation. For instance, the \(L_{2}\) SVM generally works well except when there are too many noise inputs, while the \(L_{1}\) SVM is more preferred in the presence of many noise variables. In this article we propose and explore an adaptive learning procedure called the \(Lq\) SVM, where the best \(q>0\) is automatically chosen by data. Both two- and multi-class classification problems are considered. We show that the new adaptive approach combines the benefit of a class of non-adaptive procedures and gives the best performance of this class across a variety of situations. Moreover, we observe that the proposed \(L_q\) penalty is more robust to noise variables than the \(L_{1}\) and \(L_{2}\) penalties. An iterative algorithm is suggested to solve the \(L_q\) SVM efficiently. Simulations and real data applications support the effectiveness of the proposed procedure.