Summary: Shape interpolation between the key frames of point set surfaces is considered to reuse the existing models and reduce the workload of animation design. An approach is proposed to compute the vertex deformation gradient of each point for point set surface. The vertex deformation gradient is defined by its \(k\) nearest neighbor points and a closed form solution is presented from the view of quadric energy optimization. The correspondence between two point set surfaces is established based on the deformation gradient and the moving least square method. The deformation gradient is decomposed into a rotation and a stretching component with the matrix polar factorization. Then each component is interpolated individually over the time. Shape interpolation is implemented through a global quadratic energy optimization. The approach is applied on point set surface directly. The approach does not need the topology information of the vertices and the compatible triangulation between the two surfaces in contrast to the interpolation between triangular meshes. Numerous examples show that the approach is effective to fulfill the shape interpolation between two point set surfaces.