Summary: The numerical approximation of nonsmooth solutions of the semilinear Klein-Gordon equation in the \(d\)-dimensional space, with \(d = 1\), \(2\), \(3\), is studied based on the discovery of a new cancellation structure in the equation. This cancellation structure allows us to construct a low-regularity correction of the Lie splitting method (i.e., exponential Euler method), which can significantly improve the accuracy of the numerical solutions under low-regularity conditions compared with other second-order methods. In particular, the proposed time-stepping method can have second-order convergence in the energy space under the regularity condition \((u, \partial_t u) \in L^\infty (0,T; H^{1 + \frac{d}{4}} \times H^{\frac{d}{4}})\). In one dimension, the proposed method is shown to have almost \(\frac{4}{3}\)-order convergence in \(L^\infty (0, T; H^1 \times L^2)\) for solutions in the same space, i.e., no additional regularity in the solution is required. Rigorous error estimates are presented for a fully discrete spectral method with the proposed low-regularity time-stepping scheme. The numerical experiments show that the proposed time-stepping method is much more accurate than previously proposed methods for approximating the time dynamics of nonsmooth solutions of the semilinear Klein-Gordon equation.