Feature extraction for classification problems and its application to face recognition. (English) Zbl 1140.68468
Summary: This study investigates a new method of feature extraction for classification problems. The method is based on the Independent Component Analysis (ICA). However, unlike the original ICA, one of the unsupervised learning methods, it is developed for classification problems by utilizing class information. The proposed method is an extension of our previous work on binary-class problems to multi-class problems. It treats the class labels as input features in order to produce two sets of new features: one that carries much information on the class labels and the other that is irrelevant to the class. The learning rule for this method is obtained using the stochastic gradient method to maximize the likelihood of the observed data. Among the new features, using only class-relevant ones, the dimension of the feature space can be greatly reduced in line with the principle of parsimony, resulting better generalization. This method was applied to recognize face identities and facial expressions using various databases such as the Yale, AT&T (former ORL), Color FERET face databases and so on. The performance of the proposed method was compared with those of conventional methods such as the principal component analysis, Fisher’s linear discriminant, etc. The experimental results show that the proposed method performs well for face recognition problems.
MSC:
| 68T10 | Pattern recognition, speech recognition |
References:
| [1] | M. Turk, A. Pentland, Face recognition using eigenfaces, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1991, pp. 586-591.; M. Turk, A. Pentland, Face recognition using eigenfaces, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1991, pp. 586-591. |
| [2] | Belhumeur, P.; Hespanha, J.; Kriegman, D., Eigenfaces vs. fisherfaces: recognition using class specific linear projection, IEEE Trans. Pattern Anal. Mach. Intell., 19, 7, 711-720 (1997) |
| [3] | Liu, C.; Wechsler, H., Evolutionary pursuit and its application to face recognition, IEEE Trans. Pattern Anal. Mach. Intell., 22, 6, 570-582 (2000) |
| [4] | M.-H. Yang, Face recognition using kernel methods, Adv. Neural Inf. Process. Syst. 14 (2002).; M.-H. Yang, Face recognition using kernel methods, Adv. Neural Inf. Process. Syst. 14 (2002). |
| [5] | Lu, J.; Plataniotis, N.; Venetsanopoulos, A. N., Face recognition using kernel direct discriminant analysis algorithms, IEEE Trans. Neural Networks, 14, 1, 117-126 (2003) |
| [6] | Bartlett, M. S.; Movellan, J. R.; Sejnowski, T. J., Face recognition by independent component analysis, IEEE Trans. Neural Networks, 13, 6, 1450-1464 (2002) |
| [7] | Bell, A.; Sejnowski, T., An information-maximization approach to blind separation and blind deconvolution, Neural Comput., 7, 6, 1129-1159 (1995) |
| [8] | Bartlett, M.; Sejnowski, T., Viewpoint invariant face recognition using independent component analysis and attractor networks, Neural Inf. Process. Syst. Nat. Synth., 9, 817-823 (1997) |
| [9] | Donato, G.; Bartlett, M.; Hager, J.; Ekman, P.; Sejnowski, T., Classifying facial actions, IEEE Trans. Pattern Anal. Mach. Intell., 21, 10, 974-989 (1999) |
| [10] | Kwak, N.; Choi, C.-H.; Choi, C.-Y., Feature extraction using ICA, (Proceedings of the International Conference on Artificial Neural Networks (2001), Vienna: Vienna Austria), 568-573 · Zbl 1001.68687 |
| [11] | Kwak, N.; Choi, C.-H., A new method of feature extraction and its stability, (Proceedings of the International Conference on Artificial Neural Networks (2002), Madrid: Madrid Spain), 480-485 · Zbl 1013.68641 |
| [12] | Kwak, N.; Choi, C.-H., Feature extraction based on ICA for binary classification problems, IEEE Trans. Knowl. Data Eng., 15, 6, 1374-1388 (2003) |
| [13] | N. Yukinawa, S. Oba, K. Kato, S. Ishii, Multi-class pattern classification based on a probabilistic model of combining binary classifiers, in: Proceedings of the International Conference on Artificial Neural Networks, 2005, pp. 337-342.; N. Yukinawa, S. Oba, K. Kato, S. Ishii, Multi-class pattern classification based on a probabilistic model of combining binary classifiers, in: Proceedings of the International Conference on Artificial Neural Networks, 2005, pp. 337-342. |
| [14] | Hsu, C.-W.; Lin, C.-J., A comparison of methods for multiclass support vector machines, IEEE Trans. Neural Networks, 13, 2, 415-425 (2002) |
| [15] | D. Tax, R. Duin, Using two-class classifiers for multiclass classification, in: Proceedings of the International Conference on Image Processing, 2002.; D. Tax, R. Duin, Using two-class classifiers for multiclass classification, in: Proceedings of the International Conference on Image Processing, 2002. · Zbl 1002.68599 |
| [16] | Wu, T.-F.; Weng, R., Probability estimates for multi-class classification by pairwise coupling, J. Mach. Learn. Res., 5, 979-1005 (2004) · Zbl 1222.68336 |
| [17] | Herault, J.; Jutten, C., Space or time adaptive signal processing by neural network models, (Proceedings of the AIP Conference Neural Networks Computing, vol. 151 (1986), Snowbird: Snowbird UT, USA), 206-211 |
| [18] | J. Cardoso, Source separation using higher order moments, in: Proceedings of the ICASSP, 1989, pp. 2109-2112.; J. Cardoso, Source separation using higher order moments, in: Proceedings of the ICASSP, 1989, pp. 2109-2112. |
| [19] | Comon, P., Independent component analysis a new concept?, Signal Process., 36, 287-314 (1994) · Zbl 0791.62004 |
| [20] | Obradovic, D.; Deco, G., Blind source separation: are information maximization and redundancy minimization different?, (Proceedings of the IEEE Workshop on Neural Networks for Signal Processing (1997), Florida) · Zbl 0860.68091 |
| [21] | Cardoso, J., Infomax and maximum likelihood for blind source separation, IEEE Signal Process. Lett., 4, 4, 112-114 (1997) |
| [22] | Lee, T.-W.; Girolami, M.; Bell, A.; Sejnowski, T., A unifying information—theoretic framework for independent component analysis, Comput. Math. Appl., 39, 11, 1-21 (2000) · Zbl 1054.94004 |
| [23] | Lee, T.-W.; Girolami, M.; Sejnowski, T., Independent component analysis using an extended Infomax algorithm for mixed sub-Gaussian and super-Gaussian sources, Neural Comput., 11, 2, 417-441 (1999) |
| [24] | Xu, L.; Cheung, C.; Amari, S.-I., Learned parametric mixture based ICA algorithm, Neurocomputing, 22, 1-3, 69-80 (1998) · Zbl 0911.68159 |
| [25] | Hyvarinen, A.; Oja, E.; Hoyer, P.; Hurri, J., Image feature extraction by sparse coding and independent component analysis, (Proceedings of the 14th International Conference on Pattern Recognition (1998), Brisbane: Brisbane Australia) |
| [26] | Cover, T.; Thomas, J., Elements of Information Theory (1991), Wiley: Wiley New York · Zbl 0762.94001 |
| [27] | Hyvarinen, A.; Karhunen, J.; Oja, E., Independent Component Analysis (2001), Wiley/Interscience: Wiley/Interscience New York |
| [28] | Cardoso, J., On the stability of source separation algorithms, J. VLSI Signal Process. Syst., 26, 1, 7-14 (2000) · Zbl 0965.68130 |
| [29] | Kwak, N.; Choi, C.-H.; Ahuja, N., Face recognition using feature extraction based on independent component analysis, (Proceedings of the International Conference on Image Processing (2002), Rochester: Rochester NY) |
| [30] | Micheli-Tzanakou, E., Supervised and Unsupervised Pattern Recognition (2000), CRC Press: CRC Press Boca Raton, FL |
| [31] | J. Yang, J.-Y. Yang, Is ICA significantly better than PCA for face recognition? in: Proceedings of the 10th IEEE International Conference on Computer Vision, vol. 1, 2005, pp. 198-203.; J. Yang, J.-Y. Yang, Is ICA significantly better than PCA for face recognition? in: Proceedings of the 10th IEEE International Conference on Computer Vision, vol. 1, 2005, pp. 198-203. |
| [32] | Samaria, F.; Harter, A., Parameterisation of a stochastic model for human face identification, (Proceedings of the 2nd IEEE Workshop on Applications of Computer Vision (1994), Sarasota: Sarasota FL) |
| [33] | Phillips, P.; Moon, H.; Rizvi, S.; Rauss, P., The FERET evaluation methodology for face recognition algorithms, IEEE Trans. Pattern Recognition Mach. Intell., 22, 10, 1090-1104 (2000) |
| [34] | National Institute of Standars and Technology, the Color FERET database. \( \langle;\) http://www.nist.gov/humanid/colorferet \(\rangle;\); National Institute of Standars and Technology, the Color FERET database. \( \langle;\) http://www.nist.gov/humanid/colorferet \(\rangle;\) |
| [35] | T. Sim, S. Baker, M. Bsat, The CMU pose, illumination, and expression database.; T. Sim, S. Baker, M. Bsat, The CMU pose, illumination, and expression database. |
| [36] | Lyons, M. J.; Budynek, J.; Akamatsu, S., Automatic classification of single facial images, IEEE Trans. Pattern Anal. Mach. Intell., 21, 12, 1357-1362 (1999) |
| [37] | Mitchell, T., Machine Learning (1997), McGraw-Hill: McGraw-Hill New York · Zbl 0913.68167 |
| [38] | Lu, J.; Plataniotis, K.; Venetsanopoulos, A., Regularization studies of linear discriminant analysis in small sample size scenarios with applications to face recognition, Pattern Recognition Lett., 26, 181-191 (2005) |
| [39] | Kim, C.; Choi, C.-H., Image covariance-based subspace method for face recognition, Pattern Recognition, 40, 1592-1604 (2007) · Zbl 1107.68463 |
| [40] | Cevikalp, H.; Neamtu, M.; Wilkes, M.; Barkana, A., Discriminative common vectors for face recognition, IEEE Trans. Pattern Recognition Mach. Intell., 27, 1, 4-13 (2005) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
