Summary: This paper presents a novel multiobjective endmember extraction approach for nonlinear hyperspectral mixtures by assuming that the distribution of mixtures conforms to a nonlinear manifold and the endmembers correspond to its extreme points. To identify the endmembers, the approach aims to seek a set of pixels which define a simplex with the maximum volume along the manifold. Meanwhile, several obstacles are properly settled to make it a good performance. First, calculating a simplex’s volume along the manifold needs to calculate the geodesic distance (i.e., the shortest path) between its vertices on the \(k\)-nearest neighbor (kNN) graph of the manifold data, but it is time-consuming to go through all the manifold points to search the desired simplex. Therefore, a boundary detection technique is proposed to restrict the identification of endmembers within the boundary points of the manifold to improve the time efficiency. Second, the volume of the geodesic distance based simplex is sensitive to the deviations in the geodesic distance caused by noise. To settle this issue, the multiple regression based noise estimation method is applied due to the high correlation among hundreds of spectral bands. Therefore, the spectral noise can be removed before the calculation of geodesic distance. Third, the number of endmembers is of crucial importance but hard to determine, so it is usually specified beforehand in most unmixing approaches. The proposed approach can instinctively obtain a set of simplices with the maximum volume corresponding to different numbers of endmembers, thus providing more insight for determining the optimal combination of endmembers. In addition, the proposed method is a population based optimization method which is less likely to get trapped into the local optimum. The experiments on synthetic as well as real data sets demonstrate the validity and superiority of the proposed method as compared with the methods of the same type.