Kernel quadratic discriminant analysis for small sample size problem. (English) Zbl 1140.68485
Summary: It is generally believed that Quadratic Discriminant Analysis (QDA) can better fit the data in practical pattern recognition applications compared to Linear Discriminant Analysis (LDA) method. This is due to the fact that QDA relaxes the assumption made by LDA-based methods that the covariance matrix for each class is identical. However, it still assumes that the class conditional distribution is Gaussian which is usually not the case in many real-world applications. In this paper, a novel kernel-based QDA method is proposed to further relax the Gaussian assumption by using the kernel machine technique. The proposed method solves the complex pattern recognition problem by combining the QDA solution and the kernel machine technique, and at the same time, tackles the so-called small sample size problem through a regularized estimation of the covariance matrix. Extensive experimental results indicate that the proposed method is a more sophisticated solution outperforming many traditional kernel-based learning algorithms.
MSC:
| 68T10 | Pattern recognition, speech recognition |
Keywords:
linear discriminant analysis; quadratic discriminant analysis; small sample size; kernel machine techniqueReferences:
| [1] | Belhumeur, P. N.; Hespanha, J. P.; Kriegman, D. J., Eigenfaces vs. Fisherfaces: recognition using class specific linear projection, IEEE Trans. Pattern Anal. Mach. Intell., 19, 7, 711-720 (1997) |
| [2] | Chen, L.-F.; Liao, H.-Y. M.; Ko, M.-T.; Lin, J.-C.; Yu, G.-J., A new LDA-based face recognition system which can solve the small sample size problem, Pattern Recognition, 33, 1713-1726 (2000) |
| [3] | Yu, H.; Yang, J., A direct lda algorithm for high-dimensional data with application to face recognition, Pattern Recognition, 34, 2067-2070 (2001) · Zbl 0993.68091 |
| [4] | Lu, J.; Plataniotis, K.; Venetsanopoulos, A., Regularization studies of linear discriminant analysis in small sample size scenarios with application to face recognition, Pattern Recognition Lett., 26, 2, 181-191 (2005) |
| [5] | Lu, J.; Plataniotis, K. N.; Venetsanopoulos, A. N., Face recognition using LDA based algorithms, IEEE Trans. Neural Networks, 14, 1, 195-200 (2003) |
| [6] | Lu, J.; Plataniotis, K. N.; Venetsanopoulos, A. N., Regularized discriminant analysis for the small sample size problem in face recognition, Pattern Recognition Lett., 24, 16, 3079-3087 (2003) |
| [7] | Duda, R. O.; Hart, P. E.; Stork, D. G., Pattern Classification (2000), Wiley: Wiley New York, NY |
| [8] | Kanal, L.; Chandrasekaran, B., On dimensionality and sample size in statistical pattern classification, Pattern Recogntion, 3, 238-255 (1971) |
| [9] | Wald, P.; Kronmal, R., Discriminant functions when covariance are unequal and sample sizes are moderate, Biometrics, 33, 479-484 (1977) · Zbl 0371.62091 |
| [10] | Raudys, S. J.; Jain, A. K., Small sample size effects in statistical pattern recognition: recommendations for practitioners, IEEE Trans. Pattern Anal. Mach. Intell., 13, 3, 252-264 (1991) |
| [11] | Friedman, J. H., Regularized discriminant analysis, J. Am. Statist. Assoc., 84, 165-175 (1989) |
| [12] | Vapnik, V. N., The Nature of Statistical Learning Theory (1995), Springer: Springer New York · Zbl 0934.62009 |
| [13] | Scholkopf, B.; Burges, C.; Smola, A. J., Advances in Kernel Methods—Support Vector Learning (1999), MIT Press: MIT Press Cambridge, MA |
| [14] | Ruiz, A.; de Teruel, P. L., Nonlinear kernel-based statistical pattern analysis, IEEE Trans. Neural Networks, 12, 1, 16-32 (2001) |
| [15] | Müller, K.-R.; Mika, S.; Rätsch, G.; Tsuda, K.; Schölkopf, B., An introduction to kernel-based learning algorithms, IEEE Trans. Neural Networks, 12, 2, 181-201 (2001) |
| [16] | Shawe-Taylor, J.; Cristianini, N., Kernel Methods for Pattern Analysis (2004), Cambridge University Press: Cambridge University Press Cambridge |
| [17] | Lu, J.; Plataniotis, K. N.; Venetsanopoulos, A. N., Face recognition using kernel direct discriminant analysis algorithms, IEEE Trans. Neural Networks, 14, 1, 117-126 (2003) |
| [18] | Phillips, P. J.; Wechsler, H.; Huang, J.; Rauss, P., The FERET database and evaluation procedure for face recognition algorithms, Image Vision Comput. J., 16, 5, 295-306 (1998) |
| [19] | Phillips, P. J.; Moon, H.; Rizvi, S. A.; Rauss, P. J., The FERET evaluation methodology for face-recognition algorithms, IEEE Trans. Pattern Anal. Mach. Intell., 22, 10, 1090-1104 (2000) |
| [20] | Scholkopf, B.; Smola, A.; Muller, K., Nonlinear component analysis as a kernel eigenvalue problem, Neural Comput., 10, 1299-1319 (1999) |
| [21] | Baudat, G.; Anouar, F., Generalized discriminant analysis using a kernel approach, Neural Comput., 12, 2385-2404 (2000) |
| [22] | C. Blake, E. Keogh, C.J. Merz, uci repository of machine learning databases, Department of Information and Computer Science, University of California, Irvine, \( \langle;\) http://www.ics.uci.edu/mlearn \(\rangle;\); C. Blake, E. Keogh, C.J. Merz, uci repository of machine learning databases, Department of Information and Computer Science, University of California, Irvine, \( \langle;\) http://www.ics.uci.edu/mlearn \(\rangle;\) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
