Summary: We wish to evaluate and compare models that are non-nested and fit to data using different fitting criteria. We first estimate the parameters in all models by optimizing goodness-of-fit to a data set. Then, to assess a candidate model, we simulate a population of data sets from it and evaluate the goodness-of-fit of all the models, without re-estimating the parameter values. Finally, we see whether the vector of goodness-of-fit criteria for the original data is compatible with the multivariate distribution of these criteria for the simulated data sets. By simulating from each model in turn, we determine whether any, or several, models are consistent with the data. We apply the method to compare three models, fit at different temporal resolutions to binary time series of animal behaviour data, concluding that a semi-Markov model gives a better fit than latent Gaussian and hidden Markov models.