A novel supervised dimensionality reduction algorithm: graph-based Fisher analysis. (English) Zbl 1231.68196
Summary: In this paper, a novel supervised dimensionality reduction (DR) algorithm called graph-based Fisher analysis (GbFA) is proposed. More specifically, we redefine the intrinsic and penalty graph and trade off the importance degrees of the same-class points to the intrinsic graph and the importance degrees of the not-same-class points to the penalty graph by a strictly monotone decreasing function; then the novel feature extraction criterion based on the intrinsic and penalty graph is applied. For the non-linearly separable problems, we study the kernel extensions of GbFA with respect to positive definite kernels and indefinite kernels, respectively. In addition, experiments are provided for analyzing and illustrating our results.
MSC:
| 68T05 | Learning and adaptive systems in artificial intelligence |
| 62H25 | Factor analysis and principal components; correspondence analysis |
| 68T10 | Pattern recognition, speech recognition |
Keywords:
dimensionality reduction; intrinsic graph; penalty graph; positive definite kernels; indefinite kernelsSoftware:
KPCA plus LDAReferences:
| [1] | Jolliffe, I. T., Principal Component Analysis (2002), Springer-Verlag: Springer-Verlag New York · Zbl 1011.62064 |
| [2] | S. Mika, Kernel Fisher Discriminant, Ph.D. Thesis, University of Technology, Berlin, 2002.; S. Mika, Kernel Fisher Discriminant, Ph.D. Thesis, University of Technology, Berlin, 2002. · Zbl 1149.68413 |
| [3] | Ji, Sh.-W.; Ye, J.-P., Generalized linear discriminant analysis: a unified framework and efficientmodel selection, IEEE Transactions on Neural Networks, 19, 10, 1768-1782 (2008) |
| [4] | Lu, J.-W.; Plataniotis, K.; Venetsanopoulos, A., Regularization studies of linear discriminant analysis in small sample size scenarios with application to face recognition, Pattern Recognition Letters, 26, 181-191 (2005) |
| [5] | Hastie, T.; Buja, A.; Tibshirani, R., Penalized discriminant analysis, The Annals of Statistics, 23, 1, 73-102 (1995) · Zbl 0821.62031 |
| [6] | Howland, P.; Park, H., Generalizing discriminant analysis using the generalized singular value decomposition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 26, 8, 995-1006 (2004) |
| [7] | Ye, J.-P.; Li, Q., A two-stage linear discriminant analysis via QR-decomposition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 6, 929-941 (2005) |
| [8] | Ye, J.-P., Computational and theoretical analysis of null space and orthogonal linear discriminant analysis, Jouranl of Machine Learning Research, 7, 1183-1204 (2006) · Zbl 1222.62082 |
| [9] | Chen, L.-F.; Mark Liao, H. Y.; Ko, M. T.; Lin, J.-Ch; Yu, G.-J., A new LDA-based face recognition systerm which can solve the small sample size problem, Pattern Recognition, 33, 1713-1726 (2000) |
| [10] | Yu, H.; Yang, J., A direct LDA algorithm for high-dimensional data-with application to face recognition, Pattern Recognition, 34, 2067-2070 (2001) · Zbl 0993.68091 |
| [11] | Yang, J.; Yang, J.-Y., Why can LDA be performed in PCA transformed space?, Pattern Recognition, 36, 563-566 (2003) |
| [12] | Belhumeur, P.; Hespanha, J.; Kriegman, D., Eigenfaces vs. Fisherfaces: recognition using class specific linear projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, 19, 7, 711-720 (1997) |
| [13] | He, X.-F.; Niyogi, P., Locality preserving projections, Advances in Neural Information Processing Systems, 16 (2003), MIT Press: MIT Press Cambridge, Mass |
| [14] | Roweis, S. T.; Saul, L. K., Nonlinear dimensionality reduction by locally linear embedding, Science, 290, 5500, 2323-2326 (2000) |
| [15] | M. Sugiyama, Local Fisher discriminant analysis for supervised dimensionality reduction, in: Proceedings of the 23rd International Conference on Machine Learning, AMC Press, 2006.; M. Sugiyama, Local Fisher discriminant analysis for supervised dimensionality reduction, in: Proceedings of the 23rd International Conference on Machine Learning, AMC Press, 2006. |
| [16] | He, X.-F.; Yan, Sh.-Ch.; Hu, Y.-X.; Niyogi, P.; Zhang, H.-J, Face recognition using Laplacianfaces, IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 3, 328-340 (2005) |
| [17] | Yang, J.; Zhang, D.; Yang, J.-Y.; Niu, B., Globally maximizing, locally minimizing: unsupervised discriminant projection with applications to face and palm biometrics, IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 4, 650-664 (2007) |
| [18] | Yan, Sh.-Ch.; Xu, D.; Zhang, B.-Y.; Zhang, H.-J.; Yang, Q.; Lin, S., Graph embedding and extensions: a general framework for dimensionality reduction, IEEE Transactions on Pattern Analysis and Machine Intelligence, 29, 1, 40-51 (2007) |
| [19] | Cai, H.-P.; Mikolajczyk, K.; Matas, J., Learning linear discriminant projections for dimensionality reduction of image descriptors, IEEE Transactions on Pattern Analysis and Machine Intelligence, 33, 2, 338-352 (2011) |
| [20] | Zhang, L.-M.; Qiao, L.-Sh; Chen, S.-C., Graph-optimized locality preserving projections, Pattern Recognition, 43, 6, 1993-2002 (2010) · Zbl 1191.68611 |
| [21] | Qiao, L.-Sh; Chen, S.-C.; Tan, X.-Y., Sparsity preserving discriminant analysis for single training image face recognition, Pattern Recognition, 31, 5, 422-429 (2010) |
| [22] | Yang, B.; Chen, S.-C., Disguised discrimination of locality-based unsupervised dimensionality reduction, International Journal of Pattern Recognition and Artificial Intelligence, 24, 7, 1011-1025 (2010) |
| [23] | Alpay., D., The Schur Algorithm, Reproducing Kernel Spaces and System Theory (2001), American Mathematical Society: American Mathematical Society Providence · Zbl 1059.93001 |
| [24] | Pekalska, E.; Haasdonk, B., Kernel discriminant analysis for positive definite and indefinite kernels, IEEE Transactions on Pattern Analysis and Machine Intelligence, 31, 6, 1017-1032 (2009) |
| [25] | Haasdonk, B., Feature space interpretation of SVMs with indefinite kernels, IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 4, 482-492 (2005) |
| [26] | Yang, J.; Frangi, F.; Yang, J.-Y.; Zhang, D.; Jin, Z., KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, 27, 2, 230-244 (2005) |
| [27] | F. Chung, Spectral Graph Theory, CBMS Regional Conference Series in Mathematics, 1997.; F. Chung, Spectral Graph Theory, CBMS Regional Conference Series in Mathematics, 1997. · Zbl 0867.05046 |
| [28] | Cui, Y.; Fan, L. -Y., Comparison of KPCA transformation matrices with definite and indefinite kernels for high-dimensional data, Journal of Shandong University (Engineering Science), 41, 1, 17-23 (2011) |
| [29] | Schölkopf, B.; Herbrich, R.; Smola, A., A generalized representer theorem, Lecture Notes in Computer Science, 2111, 416-426 (2001) · Zbl 0992.68088 |
| [30] | Goldfarb, L., A new approach to pattern recognition, Pattern Recognition, 2, 241-402 (1985) · Zbl 0643.68131 |
| [31] | B. Haasdonk, E. Pekalska, Indefinite kernel Fisher discriminant, in: Proceedings of the International Conference on Pattern Recognition, 2008.; B. Haasdonk, E. Pekalska, Indefinite kernel Fisher discriminant, in: Proceedings of the International Conference on Pattern Recognition, 2008. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
