In the present work, we investigate the dynamics of nonlinear Rossby waves in zonally varying background current under generalized beta approximation. The effects of the zonally varying background current, the spatial-temporal varying topography, the potential forcing and the dissipation on nonlinear Rossby waves are all taken into consideration. We derive a new modified Korteweg–deVries equation with variable coefficients for the Rossby wave amplitude with the help of multiple scales method and perturbation expansions. Based on the obtained model equation, the physical mechanisms of nonlinear Rossby waves are analyzed. Within the present selected parameter ranges, the qualitative results demonstrate that the generalized beta and basic topography are essential factors in exciting the nonlinear Rossby solitary waves. In addition, the zonally varying flow affects the linear phase speed and the linear growth or decay characteristics of the waves. The results also show that the spatial-temporal slowly varying topography, which represents an unstable mechanism for the evolution of Rossby solitary waves, is a factor in linear growth or decay. Furthermore, to validate the efficiency of the obtained model equation, a weakly nonlinear method and numerical simulation are adopted to solve the obtained equation and the results indicate the consistency between the qualitative analysis and the quantitative solutions in explaining the present equation.