From the paper: A central problem in science is deciding among competing explanations of data containing random errors. We argue that assessing the “complexity” of explanations is essential to a theoretically well-founded model selection procedure. We forrnulate model complexity in terms of the geometry of the space of probability distributions. Geometric complexity provides a clear intuitive understanding of several extant notions of model complexity. This approach allows us to reconceptualize the model selection problem as one of counting explanations that lie close to the “truth.” We demonstrate the usefulness of the approach by applying it to the recovery of models in psychophysics.
Model selection can proceed confidently when a well-justified and intuitive framework for its central concept, complexity, is available. We have shown that the geometry of the space of probability distributions provides such a framework. Rather than including seemingly disparate measures of complexity such as the number of parameters and the functional form, we construct the geometric complexity of a model by counting the number of distributions it indexes. This quantity is manifestly invariant under reparametrizations and is a basic ingredient for assessing the complexity of a statistical model in the minimum description length and Bayesian selection methods. These tools provide powerful methods of evaluating the effectiveness of models and the relationships between them.