Summary: With the increasing demands of visual surveillance systems, human identification at a distance has gained more attention from the researchers recently. Gait analysis can be used as an unobtrusive biometric measure to identify people at a distance without any attention of the human subjects. We propose a novel effective method for both automatic viewpoint and person identification by using only the silhouette sequence of the gait. The gait silhouettes are nonlinearly transformed into low-dimensional embedding by Gaussian process latent variable model, and the temporal dynamics of the gait sequences are modeled by Hidden Markov Models (HMMs). The experimental results show that our method has higher recognition rate than the other methods.