Summary: The well-known eigenface method uses an eigenface set obtained from principal component analysis. However, the single eigenface set is not enough to represent the complicated face images with large variations of poses and/or illuminations. To overcome this weakness, we propose a second-order mixture-of-eigenfaces method that combines the second-order eigenface method [L. Wang and T. K. Tan, “A new proposal for face feature description”, ISO MPG m5750, Noordwijkerhout (March 2000)] and the mixture-of-eigenfaces method (a.k.a. Gaussian mixture model) [H. Kim, D. Kim and S. Bang, “A PCA mixture model with an efficient model selection method”, Proceedings IJCNN2001 (2001)]. In this method, we use a couple of mixtures of multiple eigenface sets: one is a mixture of multiple approximate eigenface sets for face images and another is a mixture of multiple residual eigenface sets for residual face images. Each mixture of multiple eigenface sets has been obtained from expectation maximization learning consecutively.
Based on two mixture of multiple eigenface sets, each face image is represented by a couple of feature vectors obtained by projecting the face image onto a selected approximate eigenface set and then by projecting the residual face image onto a selected residual eigenface set. Recognition is performed by the distance in the feature space between the input image and the template image stored in the face database. Simulation results show that the proposed second-order mixture-of-eigenfaces method is best for face images with illumination variations and the mixture-of-eigenfaces method is best for the face images with pose variations in terms of average of the normalized modified retrieval rank and false identification rate.