Summary: We give a definition of asymptotically locally Lifshitz spacetimes, with boundary data appropriate for a non-relativistic theory on the boundary. Solutions satisfying these boundary conditions are constructed in an asymptotic expansion. We identify the boundary data with sources for dual field theory operators, and give a prescription for calculating the one-point functions of the field theory operators (including the stress tensor) in the presence of arbitrary sources. The divergences in these one-point functions can be cancelled by holographic renormalization, adding counterterms which are local functions of the boundary data.