The focus of this paper is the effect of spatial non-uniformity in the ambient flow on the force acting on a rigid sphere when the sphere Reynold number, Re, is in the range 10 to 300. Direct numerical simulations based on a pseudospectral methodology are carried out to solve the unsteady three-dimensional flow field around a sphere which is either held stationary or allowed to translate freely under the hydrodynamic forces. The various components of the total force, namely the initial, steady viscous, and history forces, are systematically estimated in the context of linearly varying straining flows. The inertial forces are isolated by computing the rapid changes in the drag and lift forces in response to a rapid acceleration of the ambient flow. It is shown that the inertial forces arising due to convective acceleration at moderate Reynolds numbers produce the inviscid flow result. While the effect of temporal acceleration depends only on the sign and magnitude of the acceleration, the effect of convective acceleration is shown to depend also on the initial state of the ambient flow. A simple theoretical argument is presented to support numerical observations. It is also shown that the effect of convective acceleration on the steady viscous force can be realized on a slower time scale. The results show that the history kernels currently available in the literature are not adequate to represent the effect of nonuniformity on the history force.