Intrinsic credible regions: an objective Bayesian approach to interval estimation (with comments and rejoinder). (English) Zbl 1087.62036
Summary: This paper defines intrinsic credible regions, a method to produce objective Bayesian credible regions which only depend on the assumed model and the available data. Lowest posterior loss (LPL) regions are defined as Bayesian credible regions which contain values of minimum posterior expected loss; they depend both on the loss function and on the prior specification. An invariant, information theory based loss function, the intrinsic discrepancy, is argued to be appropriate for scientific communication. Intrinsic credible regions are the lowest posterior loss regions with respect to the intrinsic discrepancy loss and the appropriate reference prior. The proposed procedure is completely general, and is invariant under both reparametrization and marginalization. The exact derivation of intrinsic credible regions often requires numerical integration, but good analytical approximations are provided. Special attention is given to one-dimensional intrinsic credible intervals; their coverage properties show that they are always approximate (and sometimes exact) frequentist confidence intervals. The method is illustrated with a number of examples.
MSC:
| 62F15 | Bayesian inference |
| 62F25 | Parametric tolerance and confidence regions |
| 62B10 | Statistical aspects of information-theoretic topics |
Keywords:
amount of information; intrinsic discrepancy; Bayesian asymptotics; confidence intervals; Fisher information; HPD regions; Jeffreys priors; LPL regions; objective priors; reference priors; point estimation; probability centred intervals; region estimation; frequentist coverageReferences:
| [1] | Barnard, G. A. (1980). Pivotal inference and the Bayesian controversy (with discussion). In J. M. Bernardo, M. H. DeGroot, D. V. Lindley, and A. F. M. Smith, eds.,Bayesian Statistics 1, pp. 295–318, University Press, Valencia. · Zbl 0458.62003 |
| [2] | Berger, J. O. (2000). Bayesian analysis: A look at today and thoughts of tomorrow.Journal of the American Statististical Association, 95:1269–1276. · Zbl 1009.62532 · doi:10.2307/2669768 |
| [3] | Berger, J. O. andBernardo, J. M. (1989). Estimating a product of means: Bayesian analysis with reference priors.Journal of the American Statististical Association, 84:200–207. · Zbl 0682.62018 · doi:10.2307/2289864 |
| [4] | Berger, J. O. andBernardo, J. M. (1992a). On the development of reference priors (with discussion). In J. M. Bernardo J. O. Berger, A. P. Dawid, and A. F. M. Smith, eds.,Bayesian Statistics 4, pp. 35–60. Oxford University Press, Oxford. |
| [5] | Berger, J. O. andBernardo, J. M. (1992b). Ordered group reference priors with applications to a multinomial problem.Biometrika, 79:25–37. · Zbl 0763.62014 · doi:10.1093/biomet/79.1.25 |
| [6] | Berger, J. O. andBernardo, J. M. (1992c). Reference priors in a variance components problem In P. K. Goel and N. S. Iyengar, eds.,Bayesian Analysis in Statistics and Econometrics, pp. 323–340. Springer-Verlag, Berlin. |
| [7] | Berger, J. O. andPericchi, L. R. (1996). The intrinsic Bayes factor for model selection and prediction.Journal of the American Statististical Association, 91:109–122. · Zbl 0870.62021 · doi:10.2307/2291387 |
| [8] | Bernardo, J. M. (1979a). Expected information as expected utility.The Annals of Statistics, 7:686–690. · Zbl 0407.62002 · doi:10.1214/aos/1176344689 |
| [9] | Bernardo, J. M. (1979b). Reference posterior distributions for Bayesian inference (with discussion).Journal of the Royal Statistical Society, Series B, 41:113–147. Reprinted inBayesian Inference (N. G. Polson and G. C. Tiao, eds.), Edward Elgar, Brookfield, VT, 1995, pp. 229–263. · Zbl 0428.62004 |
| [10] | Bernardo, J. M. (1997). Non-informative priors do not exist (with discussion).Journal of Statistical Planning and Inference, 65:159–189. · doi:10.1016/S0378-3758(97)90075-6 |
| [11] | Bernardo, J. M. (2001). Un programa de síntesis para la enseñanza universitaria de la estadística matemática contemporánea.Revista de la Real Academia de Ciencias Exactas, Físicas y Naturales (España), 95(1–2):87–105. |
| [12] | Bernardo, J. M. (2005a). Intrinsic point estimation of the normal variance. In S. K. Upadhyay, U. Singh, and D. K. Dey, eds.,Bayesian Statistics and its Applications. Anamaya Pub, New Delhi. In press. |
| [13] | Bernardo, J. M. (2005b). Reference analysis. In D. Dey and C. R. Rao, eds.,Bayesian Thinking, Modeling and Computation, vol. 25 ofHandbook of Statistics. North Holland, Amsterdam. In press |
| [14] | Bernardo, J. M. andJuárez, M. (2003). Intrinsic estimation. In J. M. Bernardo, M. J. Bayarri, J. O. Berger, A. P. Dawid, D. Heckerman, A. F. M. Smith, and M. West, eds.,Bayesian Statistics 7, pp. 465–476. Oxford University Press, Oxford. |
| [15] | Bernardo, J. M. andRamón, J. M. (1998). An introduction to Bayesian reference analysis: Inference on the ration of multinomial parameters.Journal of the Royal Statistical Society, Series D. The Statistician, 47:1–35. · doi:10.1111/1467-9884.00112 |
| [16] | Bernardo, J. M. andRueda, R. (2002). Bayesian hypothesis testing: A reference approach.International Statistical Review, 70:351–372. · Zbl 1211.62011 · doi:10.1111/j.1751-5823.2002.tb00175.x |
| [17] | Bernardo, J. M. andSmith, A. F. M. (1994).Bayesian Theory, John Wiley & Sons, Chichester. |
| [18] | Blyth, C. R. (1986). Approximate binomial confidence limits.Journal of the American Statististical Association, 81:843–855. · Zbl 0655.62029 · doi:10.2307/2289018 |
| [19] | Blyth, C. R. andStill, H. A. (1983). Binomial confidence intervals.Journal of the American Statististical Association, 78:108–116. · Zbl 0503.62028 · doi:10.2307/2287116 |
| [20] | Brown, L. (1968). Inadmissibility of the usual estimators of scale parameters in problems with unknown location and scale parameters.Annals of Mathematics and Statistics 39:29–48. · Zbl 0162.49901 · doi:10.1214/aoms/1177698503 |
| [21] | Brown, L. D., Cai, T. T., andDasGupta, A. (2001). Interval estimation for a binomial proportion (with discussion).Statistical Science, 16:101–133. · Zbl 1059.62533 |
| [22] | Brown, L. D., Cai, T. T., andDasGupta, A. (2002). Confidence intervals for a binomial proportion and Edgeworth expansion.The Annals of Statistics, 30(1):160–201. · Zbl 1012.62026 · doi:10.1214/aos/1015362189 |
| [23] | Burdick, R. K. andGraybill, F. A. (1992).Confidence Intervals on Variance Components. Marcel Dekker, New York. · Zbl 0755.62055 |
| [24] | Cano, J. A., Hernández, A., andMoreno, E. (1987). A note on maximized likelihood sets.European Journal of Operational Research, 32:291–293. · Zbl 0634.62002 · doi:10.1016/S0377-2217(87)80151-0 |
| [25] | Casella, G. (1986). Refining binomial confidence intervals.The Canadian Journal of Statistics, 14:113–129. · Zbl 0592.62029 · doi:10.2307/3314658 |
| [26] | Casella, G., Hwang, J. T., andRobert, C. P. (1993). A paradox in decision-theoretic set estimation.Statistica Sinica, 3:141–155. · Zbl 0823.62003 |
| [27] | Casella, G., Hwang, J. T. andRobert, C. P. (1994). Loss function for set estimation. In J. O. Berger and S. S. Gupta, eds.,Statistical Decision Theory and Related Topics, V, pp. 237–252. Springer Verlag, New York. |
| [28] | Datta, G. S. andMukerjee, R. (2004).Probability Matching Priors: Higher Order Asymptotics vol. 178 ofLecture Notes in Statistics. Springer Verlag, New York. · Zbl 1044.62031 |
| [29] | Data, G. S. andSweeting, T. (2005). Probability matching priors. In D. Dey and C. R. Rao, eds.,Bayesian Thinking, Modeling and Computation, vol. 25 ofHandbook of Statistics, North Holland, Amsterdam. In press. |
| [30] | de Finetti, B. (1970).Teoria delle Probabilità. Einaudi, Turin, English translation asTheory of Probability in 1974. John Wiley & Sons, Chichester. |
| [31] | Eberly, L. E. andCasella, G. (2003). Estimating Bayesian credible intervals.Journal of Statistical Planning and Inference, 112:115–132. · Zbl 1032.62023 · doi:10.1016/S0378-3758(02)00327-0 |
| [32] | Efron, B. (1987). Better bootstrap confidence intervals (with discussion).Journal of the American Statististical Association, 82:171–200. · Zbl 0622.62039 · doi:10.2307/2289144 |
| [33] | George, E. I. andMcCulloch, R. E. (1993). On obtaining invariant prior distributions.Journal of Statistical Planning and Inference, 37:169–179. · Zbl 0798.62008 · doi:10.1016/0378-3758(93)90086-L |
| [34] | Geyer, C. J. andMeeden, G. D. (2005) Fuzzy and randomized confidence intervals andp-values (with discussion).Statistical Science. To appear. |
| [35] | Gleser, L. andHwang, J. T. (1987). The nonexistence of 100(1) confidence sets of finite expected diameter in errors-in-variable and related models.The Annals of Statistics, 15:1351–1362. · Zbl 0638.62035 · doi:10.1214/aos/1176350597 |
| [36] | Granger, C. andLin, J.-L. (1994). Using the mutual information coefficient to identify lags in nonlinear models.The Journal of Time series Analysis, 15(4):371–384. · Zbl 0807.62067 · doi:10.1111/j.1467-9892.1994.tb00200.x |
| [37] | Gutiérrez-Peña, E. (1992). Expected logarithmic divergence for exponential families. In J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith, eds.,Bayesian Statistics 4, pp. 669–674. Oxford University Press, Oxford. |
| [38] | Guttman, I. (1970).Statistical Tolerance Regions: Classical and Bayesian. Griffin, London. · Zbl 0231.62052 |
| [39] | Hahn, G. J. andMeeker, W. Q. (1991).Statistical Intervals. John Wiley & Sons, Chichester. |
| [40] | Hartigan, J. A. (1966). Note on the confidence prior of Welch and Peers.Journal of the Royal Statistical Society. Series B, 28:55–56. · Zbl 0147.18401 |
| [41] | Hartigan, J. A. (1983).Bayes Theory. Springer-Verlag. Berlin. · Zbl 0537.62007 |
| [42] | Joe, H. (1989). Relative entropy measures of multivariate dependence.The Journal of American Statistical Association, 405:157–164. · Zbl 0677.62054 · doi:10.2307/2289859 |
| [43] | Juárez, M. (2004).Métodos Bayesianos Objetivos de Estimación y Contraste de Hipótesis. PhD Thesis, Universidad de Valencia, Spain. |
| [44] | Kullback, S. (1968).Information Theory and Statistics. Dover, New York, 2nd ed. Reprinted in 1978, Peter Smith, Gloucester, MA. · Zbl 0274.62036 |
| [45] | Kullback, S. andLeibler, R. A. (1951). On information and sufficiency.Annals of Mathematics and Statistics, 22:79–86. · Zbl 0042.38403 · doi:10.1214/aoms/1177729694 |
| [46] | Lindley, D. V. (1958). Fiducial distribution and Bayes’ Theorem.Journal of the Royal Statistical Society. Series B, 20:102–107. · Zbl 0085.35503 |
| [47] | Lindley, D. V. (1970).Introduction to Probability and Statistics from a Bayesian Viewpoint. Part 2: Inference, Cambridge University Press, Cambridge. · Zbl 0123.34505 |
| [48] | Peña, D. andvan der Linde, A. (2005). General measures of variability and dependence for multivariate continuous distribution. Manuscript submitted for publication. · Zbl 1315.62055 |
| [49] | Piccinato, L. (1984). A Bayesian property of the likelihood sets.Statistica, 3:367–372. · Zbl 0561.62002 |
| [50] | Robert, C. P. (1996). Intrinsic loss functionsTheory and Decision, 40:192–214. · Zbl 0848.90010 · doi:10.1007/BF00133173 |
| [51] | Robert, C. P. (2001).The Bayesian Choice, Springer Verlag, New York, 2nd ed. · Zbl 0980.62005 |
| [52] | Robert, C. P. andCasella, G. (1994). Distance penalized losses for testing and confidence set evaluation.Test, 3(1):163–182. · Zbl 0815.62008 · doi:10.1007/BF02562679 |
| [53] | Robinson, P. (1991). Consistent nonparametric entropy-based testing.Review of Economic Studies, 58:437–453. · Zbl 0719.62055 · doi:10.2307/2298005 |
| [54] | Rousseau, J. (1997). Expansions of penalized likelihood ratio statistics and consequences on matching priors for HPD regions. Tech. rep., CREST, INSEE, Paris. |
| [55] | Savage, L. J. (1954).The Foundations of Statistics. John Wiley & Sons, New York. Second edition in 1972, Dover, New York. · Zbl 0055.12604 |
| [56] | Schervish, M. J. (1995).Theory of Statistics, Springer-Verlag, Berlin. · Zbl 0834.62002 |
| [57] | Wasserman, L. (1989). A robust Bayesian interpretation of likelihood regions.The Annals of Statistics, 17:1387–1393. · Zbl 0681.62006 · doi:10.1214/aos/1176347277 |
| [58] | Welch, B. L. andPeers, H. W. (1963). On formulae for confidence points based on intervals of weighted likelihoods.Journal of the Royal Statistical Society. Series B, 25:318–329. · Zbl 0117.14205 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
