Summary: We present a new class of exponential integrators for ordinary differential equations: locally exact modifications of known numerical schemes. Local exactness means that they preserve the linearization of the original system at every point. In particular, locally exact integrators preserve all fixed points and are A-stable. We apply this approach to popular schemes including Euler schemes, the implicit midpoint rule, and the trapezoidal rule. We found locally exact modifications of discrete gradient schemes (for symmetric discrete gradients and coordinate increment discrete gradients) preserving their main geometric property: exact conservation of the energy integral (for arbitrary multidimensional Hamiltonian systems in canonical coordinates). Numerical experiments for a two-dimensional anharmonic oscillator show that locally exact schemes have very good accuracy in the neighbourhood of stable equilibrium, much higher than suggested by the order of new schemes (locally exact modification sometimes increases the order but in many cases leaves it unchanged).