Fisher’s disjunction as the principle vindicating \(p\)-values, confidence intervals, and their generalizations: a frequentist semantics for possibility theory. (English) Zbl 07698067
Summary: Null hypothesis significance testing is generalized by controlling the Type I error rate conditional on the existence of a non-empty confidence interval. The control of that conditional error rate corrects p-values by transforming them into c-values. A further generalization from point null hypotheses to composite hypotheses generates possibility measures called C-values. The framework has implications for the following areas of application in addition to that of bounded parameter spaces. First, C-values of unspecified catch-all hypotheses provide conditions under which the entire statistical model would be rejected. Second, the C-value of a point estimate or confidence interval from a previous study determines whether the conclusion of the study is replicated, discredited, or neither replicated nor discredited by a new study. Third, c-values of a finite number of hypotheses, theories, or other models facilitate both incorporating previous information into frequentist hypothesis testing and the comparison of scientific models such as those of molecular evolution. In all cases, the corrections of p-values are simple enough to be performed on a handheld device.
MSC:
| 68T37 | Reasoning under uncertainty in the context of artificial intelligence |
Keywords:
effect-size estimation; empty confidence interval; frequentist model selection; molecular clock hypothesis; replication crisis; restricted parameter spaceSoftware:
CMLReferences:
| [1] | Aitchison, J.; Silvey, S. D., Maximum-likelihood estimation of parameters subject to restraints, Ann. Math. Stat., 29, 813-828 (1958) · Zbl 0092.36704 |
| [2] | Ball, F.; Britton, T.; O’Neill, P., Empty confidence sets for epidemics, branching processes and Brownian motion, Biometrika, 89, 211-224 (2002) · Zbl 0997.62059 |
| [3] | Benjamini, Y.; Veaux, R. D.D.; Efron, B.; Evans, S.; Glickman, M.; Graubard, B. I.; He, X.; Meng, X. L.; Reid, N.; Stigler, S. M.; Vardeman, S. B.; Wikle, C. K.; Wright, T.; Young, L. J.; Kafadar, K., The ASA president’s task force statement on statistical significance and replicability, Ann. Appl. Stat., 15, 1084-1085 (2021) · Zbl 1478.62008 |
| [4] | Bickel, D. R., A frequentist framework of inductive reasoning, Sankhya A, 74, 141-169 (2012) · Zbl 1281.62027 |
| [5] | Bickel, D. R., Self-consistent confidence sets and tests of composite hypotheses applicable to restricted parameters (2014), University of Ottawa, deposited in uO Research at |
| [6] | Bickel, D. R., Bayesian revision of a prior given prior-data conflict, expert opinion, or a similar insight: a large-deviation approach, Statistics, 52, 552-570 (2018) · Zbl 1401.62036 |
| [7] | Bickel, D. R., Genomics Data Analysis: False Discovery Rates and Empirical Bayes Methods (2019), Chapman and Hall/CRC: Chapman and Hall/CRC New York |
| [8] | Bickel, D. R., Coherent checking and updating of Bayesian models without specifying the model space: a decision-theoretic semantics for possibility theory, Int. J. Approx. Reason., 142, 81-93 (2022) · Zbl 07478939 |
| [9] | Bickel, D. R., Maximum entropy derived and generalized under idempotent probability to address Bayes-frequentist uncertainty and model revision uncertainty: an information-theoretic semantics for possibility theory, Fuzzy Sets and Systems (2022) |
| [10] | Bickel, D. R., Phylogenetic Trees and Molecular Evolution: A Hands-on Introduction with Uncertainty Quantification Corrected (2022), Springer: Springer New York |
| [11] | Bickel, D. R.; Patriota, A. G., Self-consistent confidence sets and tests of composite hypotheses applicable to restricted parameters, Bernoulli, 25, 47-74 (2019) · Zbl 1512.62030 |
| [12] | Bickel, D. R.; Rahal, A., Model fusion and multiple testing in the likelihood paradigm: shrinkage and evidence supporting a point null hypothesis, Statistics, 53, 1187-1209 (2019) · Zbl 1435.62291 |
| [13] | Bromham, L., Six impossible things before breakfast: assumptions, models, and belief in molecular dating, Trends Ecol. Evol., 34, 474-486 (2019) |
| [14] | De Cooman, G., Possibility theory I: the measure- and integral-theoretic groundwork, Int. J. Gen. Syst., 25, 291-323 (1997) · Zbl 0955.28012 |
| [15] | De Cooman, G., Possibility theory II: conditional possibility, Int. J. Gen. Syst., 25, 325-351 (1997) · Zbl 0955.28013 |
| [16] | Dubois, D.; Foulloy, L.; Mauris, G.; Prade, H., Probability-possibility transformations, triangular fuzzy sets, and probabilistic inequalities, Reliab. Comput., 10, 273-297 (2004) · Zbl 1043.60003 |
| [17] | Dubois, D.; Moral, S.; Prade, H., A semantics for possibility theory based on likelihoods, J. Math. Anal. Appl., 205, 359-380 (1997) · Zbl 0884.03017 |
| [18] | Dubois, D.; Prade, H., Possibility Theory: An Approach to Computerized Processing of Uncertainty (1988), Plenum Press: Plenum Press New York · Zbl 0703.68004 |
| [19] | Dubois, D.; Prade, H.; Sandri, S., On possibility/probability transformations, (Fuzzy Logic (1993), Springer), 103-112 |
| [20] | Ermini Leaf, D.; Liu, C., Inference about constrained parameters using the elastic belief method, Int. J. Approx. Reason., 53, 709-727 (2012) · Zbl 1446.62082 |
| [21] | Fisher, R., Statistical methods and scientific induction, J. R. Stat. Soc., Ser. B, Methodol., 17, 69-78 (1955) · Zbl 0066.38008 |
| [22] | Fisher, R. A., Scientific thought and the refinement of human reasoning, J. Oper. Res. Soc. Jpn., 3, 1-10 (1960) |
| [23] | Fisher, R. A., Statistical Methods and Scientific Inference (1973), Hafner Press: Hafner Press New York · Zbl 0281.62002 |
| [24] | Fisher, R. A., Statistical Methods for Research Workers (2017), Kalpaz Publications: Kalpaz Publications Delhi, First published in 1925 · JFM 60.1162.01 |
| [25] | Fraser, D. A.S., The p-value function and statistical inference, Am. Stat., 73, 135-147 (2019) · Zbl 07588196 |
| [26] | Gabriel, K. R., Simultaneous test procedures - some theory of multiple comparisons, Ann. Math. Stat., 40, 224-250 (1969) · Zbl 0198.23602 |
| [27] | Ghasemi Hamed, M.; Serrurier, M.; Durand, N., Representing uncertainty by possibility distributions encoding confidence bands, tolerance and prediction intervals, Lecture Notes in Computer ScienceLecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 7520, 233-246 (2012) |
| [28] | Giang, P. H.; Shenoy, P. P., Decision making on the sole basis of statistical likelihood, Artif. Intell., 165, 137-163 (2005) · Zbl 1132.91371 |
| [29] | Huang, S., Primate phylogeny: molecular evidence for a pongid clade excluding humans and a prosimian clade containing tarsiers, Sci. China Life Sci., 55, 709-725 (2012) |
| [30] | Hughes, B., Psychology in Crisis (2018), Palgrave: Palgrave London |
| [31] | Infanger, D.; Schmidt-Trucksäss, A., P value functions: an underused method to present research results and to promote quantitative reasoning, Stat. Med., 38, 4189-4197 (2019) |
| [32] | Kafadar, K., Statistics and unintended consequences, Amstat News, 504, 3-4 (2019) |
| [33] | Lapointe, S.; Bobée, B., Revision of possibility distributions: a Bayesian inference pattern, Fuzzy Sets Syst., 116, 119-140 (2000) · Zbl 0966.62014 |
| [34] | Lash, T. L., The harm done to reproducibility by the culture of null hypothesis significance testing, Am. J. Epidemiol., 186, 627-635 (2017) |
| [35] | Lindley, D., Understanding Uncertainty, Wiley Series in Probability and Statistics (2013), Wiley: Wiley New York |
| [36] | Lista, L., Statistical Methods for Data Analysis in Particle Physics, Lecture Notes in Physics (2017), Springer: Springer New York · Zbl 1390.81007 |
| [37] | Mandelkern, M., Setting confidence intervals for bounded parameters, Stat. Sci., 17, 149-159 (2002) · Zbl 1013.62028 |
| [38] | Martin, R., A statistical inference course based on p-values, Am. Stat., 71, 128-136 (2017) · Zbl 07671790 |
| [39] | Martin, R.; Liu, C., Inferential models: a framework for prior-free posterior probabilistic inference, J. Am. Stat. Assoc., 108, 301-313 (2013) · Zbl 06158344 |
| [40] | Martin, R.; Liu, C., A note on p-values interpreted as plausibilities, Stat. Sin., 24, 1703-1716 (2014) · Zbl 1480.62010 |
| [41] | Masson, M. H.; Denœux, T., Inferring a possibility distribution from empirical data, Fuzzy Sets Syst., 157, 319-340 (2006) · Zbl 1083.68125 |
| [42] | Matthews, R., The p-value statement, five years on, Significance, 18, 16-19 (2021) |
| [43] | Mauris, G.; Lasserre, V.; Foulloy, L., A fuzzy approach for the expression of uncertainty in measurement, Measurement, 29, 165-177 (2001) |
| [44] | Mayo, D. G., P-value thresholds: forfeit at your peril, Eur. J. Clin. Investig., 49, Article e13170 pp. (2019) |
| [45] | Mayo, D. G.; Hand, D., Statistical significance and its critics: practicing damaging science, or damaging scientific practice?, Synthese, 200, 220 (2022) |
| [46] | Morey, R. D.; Hoekstra, R.; Rouder, J. N.; Lee, M. D.; Wagenmakers, E. J., The fallacy of placing confidence in confidence intervals, Psychon. Bull. Rev., 23, 103-123 (2016) |
| [47] | Musgrave, A., How to do without inductive logic, Science & Education, 8, 395-412 (1999) |
| [48] | Musgrave, A., The miracle argument for scientific realism, The Rutherford Journal: The New Zealand Journal for the History and Philosophy of Science and Technology, 2 (2007) |
| [49] | Musgrave, A., Experience and perceptual belief, (Parusnikova, Z.; Cohen, R., Rethinking Popper (2009), Springer: Springer New York), 5-19 |
| [50] | Neyman, J., On the two different aspects of the representative method: the method of stratified sampling and the method of purposive selection, J. R. Stat. Soc., 97, 558-625 (1934) · JFM 61.1310.02 |
| [51] | Estimating the reproducibility of psychological science, Science, 349, Article aac4716 pp. (2015) |
| [52] | Patriota, A. G., A classical measure of evidence for general null hypotheses, Fuzzy Sets Syst., 233, 74-88 (2013) · Zbl 1314.62068 |
| [53] | Patriota, A. G., On some assumptions of the null hypothesis statistical testing, Educ. Psychol. Meas., 77, 507-528 (2017) |
| [54] | Patriota, A. G., Is NHST logically flawed? Commentary on: “NHST is still logically flawed”, Scientometrics, 116, 2189-2191 (2018) |
| [55] | Puhalskii, A., Large Deviations and Idempotent Probability, Monographs and Surveys in Pure and Applied Mathematics (2001), CRC Press: CRC Press New York · Zbl 0983.60003 |
| [56] | Rubin, M., “Repeated sampling from the same population?“ A critique of Neyman and Pearson”s responses to Fisher, Eur. J. Philos. Sci., 10, 42 (2020) |
| [57] | Schneider, J. W., NHST is still logically flawed, Scientometrics, 115, 627-635 (2018) |
| [58] | Schneider, J. W., Response to commentary on “Is NHST logically flawed”, Scientometrics, 116, 2193-2194 (2018) |
| [59] | Schoenberg, R., Constrained maximum likelihood, Comput. Econ., 10, 251-266 (1997) · Zbl 0893.90179 |
| [60] | Schweder, T.; Hjort, N., Confidence, Likelihood, Probability: Statistical Inference with Confidence Distributions, Cambridge Series in Statistical and Probabilistic Mathematics (2016), Cambridge University Press: Cambridge University Press Cambridge · Zbl 1353.62007 |
| [61] | Shafer, G., A Mathematical Theory of Evidence (1976), Princeton University Press: Princeton University Press Princeton · Zbl 0359.62002 |
| [62] | Spohn, W., The Laws of Belief: Ranking Theory and Its Philosophical Applications (2012), Oxford University Press: Oxford University Press Oxford |
| [63] | Walley, P., Towards a unified theory of imprecise probability, Int. J. Approx. Reason., 24, 125-148 (2000) · Zbl 1007.28015 |
| [64] | Walley, P.; de Cooman, G., Coherence of rules for defining conditional possibility, Int. J. Approx. Reason., 21, 63-107 (1999) · Zbl 0957.68115 |
| [65] | Wang, H., Modified p-values for one-sided testing in restricted parameter spaces, Stat. Probab. Lett., 77, 625-631 (2007) · Zbl 1116.62027 |
| [66] | Wang, Z.; Klir, G. J., Generalized Measure Theory (IFSR International Series on Systems Science and Engineering) (2008), Springer: Springer New York |
| [67] | Wasserstein, R. L.; Lazar, N. A., The ASA’s statement on p-values: context, process, and purpose, Am. Stat., 70, 129-133 (2016) · Zbl 07665862 |
| [68] | Wasserstein, R. L.; Schirm, A. L.; Lazar, N. A., Moving to a world beyond “p < 0.05”, Am. Stat., 73, 1-19 (2019) · Zbl 07588180 |
| [69] | Wellek, S., A critical evaluation of the current “p-value controversy”, Biom. J., 59, 854-872 (2017) · Zbl 1391.62254 |
| [70] | Zadeh, L., Fuzzy sets as a basis for a theory of possibility, Fuzzy Sets Syst., 1, 3-28 (1978) · Zbl 0377.04002 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
