Interpretations of Probability and Bayesian Inference—an Overview

In this article, I first give a short outline of the different interpretations of the concept of probability that emerged in the twentieth century. In what follows, I give an overview of the main problems and problematic concepts from the philosophy of probability and show how they relate to Bayesian inference. In this overview, I emphasise that the understanding of the main concepts related to different interpretations of probability influences the understanding and status of Bayesian inference. In the conclusion, I express that, from a broad epistemological point of view, a kind of compatibilism between the two main lines of interpretations of probability is worth pursuing, as they represent different aspects of the epistemological process.

1. Here is a list of a few major works that deal with different aspects of Bayesian epistemology for illustration purpose: Bovens and Hartmann (2003), Earman (1991), Howson and Urbach (1999), Jaynes (2003), Jeffrey (1992), Joyce (1999), Maher (1993), and Skyrms (1990).

2. Poisson introduced the terms chance and probabilité, which are today in use in the English speaking world; a century later, Carnap used the terms ‘probability1’ and ‘probability2’, and later inductive and statistical probability.

3. This is the famous motto from his late work Theory of Probability (1974).

4. Coherence, as introduced, by de Finetti is at the same time the necessary and sufficient condition for the axioms of probability of a finite set of events to hold: ‘The result that coherence is, for a field of events, equivalent to the rules of finitely additive probability is sometimes called Ramsey-de Finetti theorem’ (von Plato 1994, p. 272).

5. ‘A full account of the Bayesian revival is still to be written. Yet, one thing seems clear. The post-war context in which Bayesianism began to blossom that differed significantly from the inimical environment of the 1930s. In consequence, the Bayesian position that emerged during the 1950s differed from what counted as a Bayesian view before the war. Neo-Bayesians such as Savage regarded as central the idea of ‘personalistic’ probability, and for the foundations of the subjective view turned not to Jeffreys but to the accounts of Bruno de Finetti and Frank Ramsey’ (Howie 2002, p. 227).

6. Professor of Biostatistics at the University of Ljubljana, Andrej Blejec, recounted a telling episode from a statistical conference in Scandinavia around the turn of the millenium. After accidentally arriving late, he entered the conference room to witness infuriated statisticians split into two camps shouting over each other. They were Bayesians and frequentists. According to Prof. Blejec, this split among statisticians seems to be mitigated by the emergence of big data, but this is a subjective recollection. Mayo (2018) writes about statistics wars, which are supposed to have ended, but according to her, they persist below the surface.

7. The two examples of non-statistical hypotheses chosen have historical roots: The first was actually the first scientific hypothesis to be successfully supported by means of Bayesian methods, while the latter has been subject of debates concerning Bayesian confirmation; Pascal’s wager, one of the early historical examples of probabilistic reasoning in Western culture, is a first version of it.

8. See, for example, the entry on inductive logic by Hawthorne (2018) in the Stanford Encyclopaedia of Philosophy: ‘This article will focus on the kind of the approach to inductive logic most widely studied by epistemologists and logicians in recent years. This approach employs conditional probability functions to represent measures of the degree to which evidence statements support hypotheses. […] Thus, this approach to the logic of evidential support is often called a Bayesian Inductive Logic or a Bayesian Confirmation Theory.’

9. In the logical Bayesian context, inference to the best explanation (IBE) is often discussed with regard to prior probabilities and to subjective/objective Bayesianism. While the tendency is to argue for some kind of compatibilism between IBE and subjective Bayesianism, Weisberg (2009) argues that IBE is not compatible with subjective Bayesianism because it violates conditionalisation. In contrast, according to him, it is compatible with objective Bayesianism as a way of informing prior probabilities and it does so by taking into account criteria such as simplicity, elegance, unification, and stability.

10. For the sake of clarity, I will avoid the term ‘classical probability’ and use ‘combinatorial probability’ instead as the reader could confuse it with classical statistics. The latter does not employ the classical concept of probability, but the modern frequentist concept of probability.

11. E.T. Jaynes, for example, identified four general principles that constrain prior probabilities: group invariance, maximum entropy, marginalization, and coding theory, but he did not consider the list exhaustive.

12. The technique is not new, but it was not so widespread until recently. In recent decades, many doctoral theses have been written on the topic, which may be a sign of changing perception about scientific approach in the scientific community. The original reference for lucky imaging is the article by Hufnagel (1966): ‘Restoration of Atmospherically Degraded Images: Woods Hole Summer Study’.

I would like to thank one of the anonymous reviewers for the thorough engagement and all the fruitful remarks and criticism.

Authors and Affiliations

1. Filozofska fakulteta, Univerza v Ljubljani, Ljubljana, Slovenia

   Peter Lukan

Corresponding author

Correspondence to Peter Lukan.

Conflict of Interest

The author declares no conflict of interest.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Reprints and permissions