Summary: In this study, a novel M-estimation based sparse grid quadrature filter (MSGQF) is proposed to improve the robust performance of the nonlinear system. We present a systematic formulation of the sparse grid quadrature filter (SGQF), and extend it to the discrete-time nonlinear system with abnormal measurement values. The M-estimation method is introduced in the SGQF, which uses the Huber’s cost function to update the measurement covariance. Convergence on the modified robust SGQF is established and proved. The sufficient conditions are shown to ensure stochastic stability of the MSGQF. A target tracking problem has been conducted to demonstrate the accuracy of the MSGQF. When measurement abnormal values appear, it outperforms the unscented Kalman filter (UKF), the cubature Kalman filter (CKF) and the SGQF. Theoretical analysis and simulation results prove that the MSGQF provides significant performance improvement in the robustness of the nonlinear system.