Summary: Numerical simulations are used to investigate the influence of heterogeneity in grain-boundary diffusivity and sliding resistance on the creep response of a polycrystal. We model a polycrystal as a two-dimensional assembly of elastic grains, separated by sharp grain boundaries. The crystal deforms plastically by stress driven mass transport along the grain boundaries, together with grain-boundary sliding. Heterogeneity is idealized by assigning each grain boundary one of two possible values of diffusivity and sliding viscosity. We compute steady state and transient creep rates as functions of the diffusivity mismatch and relative fractions of grain boundaries with fast and slow diffusion. In addition, our results show that under transient conditions, flux divergences develop at the intersection between grain boundaries with fast and slow diffusivity, which generate high local stress concentrations. The stress concentrations develop at a rate determined by the fast diffusion coefficient, and subsequently relax at a rate determined by the slow diffusion coefficient. The influence of the mismatch in diffusion coefficient, loading conditions, and material properties on the magnitude of this stress concentration is investigated in detail using a simple model problem with a planar grain boundary. The strain energy associated with these stress concentrations also makes a small fraction of the plastic strain due to diffusion and sliding recoverable on unloading. We discuss the implications of these results for conventional polycrystalline solids at high temperatures and for nanostructured materials where grain-boundary diffusion becomes one of the primary inelastic deformation mechanisms even at room temperature.