×

Bernoulli’s golden theorem in retrospect: error probabilities and trustworthy evidence. (English) Zbl 1528.60008

Summary: Bernoulli’s 1713 golden theorem is viewed retrospectively in the context of modern model-based frequentist inference that revolves around the concept of a prespecified statistical model \(\mathcal{M}_{\theta}(\mathbf{x})\), defining the inductive premises of inference. It is argued that several widely-accepted claims relating to the golden theorem and frequentist inference are either misleading or erroneous: (a) Bernoulli solved the problem of inference ‘from probability to frequency’, and thus (b) the golden theorem cannot justify an approximate Confidence Interval (CI) for the unknown parameter \(\theta \), (c) Bernoulli identified the probability \(P(A)\) with the relative frequency \(\frac{1}{n}\sum\nolimits_{k = 1}^n {x_k }\) of event \(A\) as a result of conflating \(f(\mathbf{x}_0 |\theta)\) with \(f(\theta |\mathbf{x}_0)\), where \(\mathbf{x}_0\) denotes the observed data, and (d) the same ‘swindle’ is currently perpetrated by the \(p\) value testers. In interrogating the claims (a)–(d), the paper raises several foundational issues that are particularly relevant for statistical induction as it relates to the current discussions on the replication crises and the trustworthiness of empirical evidence, arguing that: [i] The alleged Bernoulli swindle is grounded in the unwarranted claim \(\hat{\theta}_n (\mathbf{x}_0) \simeq \theta^* \), for a large enough \(n\), where \(\hat{\theta}_n (\mathbf{X})\) is an optimal estimator of the true value \(\theta^*\) of \(\theta \). [ii] Frequentist error probabilities are not conditional on hypotheses (\(H_0\) and \(H_1\)) framed in terms of an unknown parameter \(\theta\) since \(\theta\) is neither a random variable nor an event. [iii] The direct versus inverse inference problem is a contrived and misplaced charge since neither conditional distribution \(f(\mathbf{x}_0 |\theta)\) and \(f(\theta |\mathbf{x}_0 )\) exists (formally or logically) in model-based \((\mathcal{M}_{\theta} (\mathbf{x}))\) frequentist inference.

MSC:

60A05 Axioms; other general questions in probability

Software:

bootstrap
Full Text: DOI

References:

[1] Bayes, T., An essay towards solving a problem in the doctrine of chances, Philospical Transactions of the Royal Society of London, 53, 370-402 (1764)
[2] Berger, JO; Wolpert, RW, The likelihood principle. Lecture notes—Monograph series (1988), Institute of Mathematical Statistics
[3] Bernoulli, J. (1713/2006). The art of conjecturing. JHU Press.
[4] Billingsley, P., Probability and measure (1995), Wiley · Zbl 0822.60002
[5] Cohen, J., Statistical power analysis for the behavioral sciences (1988), Lawrence Erlbaum · Zbl 0747.62110
[6] Cohen, J., The earth is round (p <. 05), American Psychologist, 49, 997-1003 (1994)
[7] De Moivre, A., The doctrine of chances: Or a method of calculating the probability of events in play (1738), W. Pearson
[8] Dempster, AP, New methods for reasoning towards posterior distributions based on sample data, The Annals of Mathematical Statistics, 37, 2, 355-374 (1966) · Zbl 0178.54302
[9] Diaconis, P.; Skyrms, B., Ten great ideas about chance (2018), Princeton University Press · Zbl 1386.60007
[10] Efron, B.; Tibshirani, RJ, An introduction to the bootstrap (1993), Chapman & Hall · Zbl 0835.62038
[11] Ellis, PD, The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results (2010), Cambridge University Press
[12] Fisher, RA, On the mathematical foundations of theoretical statistics, Philosophical Transactions of the Royal Society A, 222, 309-368 (1922) · JFM 48.1280.02
[13] Fisher, RA, Statistical methods for research workers (1925), Oliver and Boyd · JFM 51.0414.08
[14] Gelman, A.; Carlin, JB; Rubin, DB, Bayesian data analysis (2004), Chapman & Hall · Zbl 1039.62018
[15] Ghosh, KJ; Delampady, M.; Samata, T., Introduction to Bayesian analysis (2006), Springer · Zbl 1135.62002
[16] Gorroochurn, P., Classic problems of probability (2012), Wiley · Zbl 1269.60005
[17] Hacking, I., Salmon’s vindication of induction, The Journal of Philosophy, 62, 10, 260-266 (1965)
[18] Hacking, I.; Mellor, D., The theory of probable inference: Neyman, Peirce and Braithwaite, Science, belief and behavior: Essays in honour of Richard B, 141-160 (1980), Cambridge University Press, Cambridge · Zbl 0479.03002
[19] Hald, A., A history of mathematical statistics from 1750 to 1930 (1998), Wiley · Zbl 0979.01012
[20] Hald, A., A history of parametric statistical inference from Bernoulli to Fisher, 1713-1935 (2007), Springer
[21] Henderson, L. (2020). The problem of induction. The Stanford Encyclopedia of Philosophy. Edward N. Zalta (ed.). https://plato.stanford.edu/archives/spr2020/entries/induction-problem/.
[22] Howson, C.; Urbach, P., Scientific rasoning: The Bayesian approach (2006), Open Court
[23] Hume, D. (1748). An Enquiry Concerning Human Understanding. Oxford: Oxford University Press.
[24] Kolmogorov, A. N. (1933). Foundations of the theory of Probability, 2nd English edition, NY: Chelsea Publishing Co.
[25] Laplace, PS, Théorie analytique des Probabilités (1812), Courcier Imprimeur
[26] Le Cam, L., A note on metastatistics or ‘an essay toward stating a problem in the doctrine of chances”, Synthese, 36, 133-160 (1977) · Zbl 0367.62005
[27] Le Cam, L., Asymptotic methods in statistical decision theory (1986), Springer · Zbl 0605.62002
[28] Lehmann, EL; Romano, JP, Testing statistical hypotheses (2005), Springer · Zbl 1076.62018
[29] Lindley, DV, Introduction to probability and statistics from the bayesian viewpoint (1965), Cambridge University Press · Zbl 0123.34505
[30] Mayo, DG; Spanos, A., Severe testing as a basic concept in a Neyman-Pearson philosophy of induction, British Journal for the Philosophy of Science, 57, 323-357 (2006) · Zbl 1098.03030
[31] Mayo, DG; Spanos, A.; Gabbay, D.; Thagard, P.; Woods, J., Error statistics, The handbook of philosophy of science, 151-196 (2011), Amsterdam: Elsevier, Amsterdam
[32] Neyman, J., Outline of a theory of statistical estimation based on the classical theory of probability, Philosophical Transactions of the Royal Statistical Society of London, A, 236, 333-380 (1937) · Zbl 0017.12403
[33] Neyman, J.; Pearson, ES, On the problem of the most efficient tests of statistical hypotheses, Philosophical Transactions of the Royal Statistical Society of London, A, 231, 289-337 (1933) · Zbl 0006.26804
[34] Nickerson, RS, Null hypothesis significance testing: A review of an old and continuing controversy, Psychological Methods, 5, 2, 241-301 (2000)
[35] Nosek, BA; Lakens, DE, A method to increase the credibility of published results, Social Psychology, 45, 137-141 (2014)
[36] O’Hagan, A., Bayesian inference (1994), Edward Arnold · Zbl 0853.62007
[37] Renyi, A., Foundations of probability (1970), Holden-Day · Zbl 0203.49801
[38] Robert, CP, The Bayesian choice: From decision-theoretic foundations to computational implementation (2007), Springer · Zbl 1129.62003
[39] Salmon, WC, The foundations of scientific inference (1967), University of Pittsburgh Press
[40] Shiryaev, AN, Probability-1 (2016), Springer
[41] Sober, E., Evidence and evolution: The logic behind the science (2008), Cambridge University Press
[42] Spanos, A. (2006). Where do statistical models come from? Revisiting the problem of specification. In J. Rojo (Ed.), Optimality: The Second Erich L. Lehmann Symposium. Lecture Notes-Monograph Series, (Vol. 49, pp. 98-119). Institute of Mathematical Statistics. · Zbl 1268.62007
[43] Spanos, A., Is frequentist testing vulnerable to the base-rate fallacy?, Philosophy of Science, 77, 565-583 (2010)
[44] Spanos, A., A frequentist interpretation of probability for model-based inductive inference, Synthese, 190, 1555-1585 (2013) · Zbl 1284.60005
[45] Spanos, A., Who should be afraid of the Jeffreys-Lindley paradox?, Philosophy of Science, 80, 73-93 (2013)
[46] Spanos, A. (2017). Why the decision-theoretic perspective misrepresents frequentist inference. In: Advances in statistical methodologies and their applications to real problems (pp. 3-28). ISBN 978-953-51-4962-0.
[47] Spanos, A., Mis-specification testing in retrospect, Journal of Economic Surveys, 32, 2, 541-577 (2018)
[48] Spanos, A., Probability theory and statistical inference: Empirical modeling with observational data (2019), Cambridge University Press · Zbl 1422.62007
[49] Spanos, A., Revisiting noncentrality-based confidence intervals, error probabilities and estimation-based effect sizes, Journal of Mathematical Psychology, 104, 102580 (2021) · Zbl 1479.91296
[50] Spanos, A.; McGuirk, A., The model specification problem from a probabilistic reduction perspective, Journal of the American Agricultural Association, 83, 1168-1176 (2001)
[51] Spanos, A.; Mayo, DG, Error statistical modeling and inference: Where methodology meets ontology, Synthese, 192, 3533-3555 (2015) · Zbl 1360.62024
[52] Von Mises, R., Probability, statistics and truth (1928), Dover
[53] Wasserstein, RL; Schirm, AL; Lazar, NA, Moving to a world beyond p < .05, American Statistian, 73, Suppl. 1, 1-19 (2019) · Zbl 07588180
[54] Williams, D., Weighing the odds: A course in probability and statistics (2001), Cambridge University Press · Zbl 0984.62001
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.