Using MCMC chain outputs to efficiently estimate Bayes factors. (English) Zbl 1270.62155
Summary: One of the most important methodological problems in psychological research is assessing the reasonableness of null models, which typically constrain a parameter to a specific value such as zero. The Bayes factor has been recently advocated in the statistical and psychological literature as a principle means of measuring the evidence in data for various models, including those where parameters are set to specific values. Yet, it is rarely adopted in substantive research, perhaps because of the difficulties in computation. Fortunately, for this problem, the Savage-Dickey density ratio [J.M. Dickey and B.P. Lientz, Ann. Math. Stat. 41, 214–226 (1970; Zbl 0188.50102)] provides a conceptually simple approach to computing the Bayes factor. We review methods for computing the Savage-Dickey density ratio, and highlight an improved method, originally suggested by A.E. Gelfand and A.F.M. Smith [J. Am. Stat. Assoc. 85, No. 410, 398–409 (1990; Zbl 0702.62020)] and advocated by S. Chib [J. Am. Stat. Assoc. 90, No. 432, 1313–1321 (1995; Zbl 0868.62027)], that outperforms those currently discussed in the psychological literature. The improved method is based on conditional quantities, which may be integrated by Markov chain Monte Carlo sampling to estimate Bayes factors. These conditional quantities efficiently utilize all the information in the MCMC chains, leading to accurate estimation of Bayes factors. We demonstrate the method by computing Bayes factors in one-sample and one-way designs, and show how it may be implemented in WinBUGS.
MSC:
| 62P15 | Applications of statistics to psychology |
| 65C40 | Numerical analysis or methods applied to Markov chains |
| 65C60 | Computational problems in statistics (MSC2010) |
| 62-04 | Software, source code, etc. for problems pertaining to statistics |
Keywords:
Bayesian statistics; Bayesian inference; Markov chain Monte Carlo; ANOVA; encompassing priorsReferences:
| [1] | Berger, J. O.; Sellke, T., Testing a point null hypothesis: the irreconcilability of p values and evidence, Journal of the American Statistical Association, 82, 397, 112-122 (1987), URL: http://www.jstor.org/stable/2289131 · Zbl 0612.62022 |
| [2] | Bernardo, J. M.; Smith, A. F.M., Bayesian theory (2000), John Wiley & Sons: John Wiley & Sons Chichester, England · Zbl 0943.62009 |
| [3] | Blackwell, D., Conditional expectation and unbiased sequential estimation, The Annals of Mathematical Statistics, 18, 105-110 (1947) · Zbl 0033.07603 |
| [4] | Chen, M.-H., Importance-weighted marginal Bayesian posterior density estimation, Journal of the American Statistical Association, 89, 427, 818-824 (1994), URL: http://www.jstor.org/stable/2290907 · Zbl 0804.62040 |
| [5] | Chen, M.-H., & Shao, Q.-M. (1997). Performance study of marginal posterior density estimation via Kullback-Leibler divergence. Test 6, 321-350.; Chen, M.-H., & Shao, Q.-M. (1997). Performance study of marginal posterior density estimation via Kullback-Leibler divergence. Test 6, 321-350. · Zbl 0905.62021 |
| [6] | Chib, S., Marginal likelihood from the Gibbs output, Journal of the American Statistical Association, 90, 1313-1321 (1995), URL: http://www.jstor.org/stable/2291521 · Zbl 0868.62027 |
| [7] | Dickey, J. M., The weighted likelihood ratio, linear hypotheses on normal location parameters, The Annals of Mathematical Statistics, 42, 1, 204-223 (1971), URL: http://www.jstor.org/stable/2958475 · Zbl 0274.62020 |
| [8] | Dickey, J. M.; Lientz, B. P., The weighted likelihood ratio, sharp hypotheses about chances, the order of a Markov chain, The Annals of Mathematical Statistics, 41, 1, 214-226 (1970), URL: http://www.jstor.org/stable/2239734 · Zbl 0188.50102 |
| [9] | Edwards, W.; Lindman, H.; Savage, L. J., Bayesian statistical inference for psychological research, Psychological Review, 70, 193-242 (1963) · Zbl 0173.22004 |
| [10] | Gelfand, A.; Smith, A. F.M., Sampling based approaches to calculating marginal densities, Journal of the American Statistical Association, 85, 398-409 (1990) · Zbl 0702.62020 |
| [11] | Gelman, A.; Carlin, J. B.; Stern, H. S.; Rubin, D. B., Bayesian data analysis (2004), Chapman and Hall: Chapman and Hall London · Zbl 1039.62018 |
| [12] | Geman, S.; Geman, D., Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images, IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721-741 (1984) · Zbl 0573.62030 |
| [13] | Heidelberger, P.; Welch, P., Simulation run length control in the presence of an initial transient, Operations Research, 31, 1109-1144 (1983), URL: http://www.jstor.org/stable/170841 · Zbl 0532.65097 |
| [14] | Jeffreys, H., Theory of probability (1961), Oxford University Press: Oxford University Press New York · Zbl 0116.34904 |
| [15] | Kass, R.; Raftery, A., Bayes factors, Journal of the American Statistical Association, 90, 773-795 (1995), URL: http://www.jstor.org/stable/2291091 · Zbl 0846.62028 |
| [16] | Klugkist, I.; Kato, B.; Hoijtink, H., Bayesian model selection using encompassing priors, Statistica Neerlandica, 59, 57-69 (2005) · Zbl 1069.62026 |
| [17] | Kotz, S.; Nadarajah, S., Multivariate \(t\) distributions and their applications (2004), Cambridge University Press: Cambridge University Press Cambridge · Zbl 1100.62059 |
| [18] | Lee, M. D., How cognitive modeling can benefit from hierarchical Bayesian models, Journal of Mathematical Psychology, 55, 1-7 (2011) · Zbl 1208.91123 |
| [19] | Liang, F.; Paulo, R.; Molina, G.; Clyde, M. A.; Berger, J. O., Mixtures of g-priors for Bayesian variable selection, Journal of the American Statistical Association, 103, 410-423 (2008), URL: http://pubs.amstat.org/doi/pdf/10.1198/016214507000001337 · Zbl 1335.62026 |
| [20] | Lindley, D. V., A statistical paradox, Biometrika, 44, 187-192 (1957) · Zbl 0080.12801 |
| [21] | Lunn, D.; Thomas, A.; Best, N.; Spiegelhalter, D., WinBUGS-a Bayesian modelling framework: concepts, structure, and extensibility, Statistics and Computing, 10, 325-337 (2000) |
| [22] | Meng, X.; Wong, W., Simulating ratios of normalizing constants via a simple identity: a theoretical exploration, Statistica Sinica, 6, 831-860 (1996) · Zbl 0857.62017 |
| [23] | Morey, R.D., & Rouder, J.N. (in press). Bayes factor approaches for testing interval null hypotheses. Psychological Methods.; Morey, R.D., & Rouder, J.N. (in press). Bayes factor approaches for testing interval null hypotheses. Psychological Methods. |
| [24] | Plummer, M. (2003). JAGS: a program for analysis of Bayesian graphical models using Gibbs sampling. In: Proceedings of the 3rd international workshop on distributed statistical computing; Plummer, M. (2003). JAGS: a program for analysis of Bayesian graphical models using Gibbs sampling. In: Proceedings of the 3rd international workshop on distributed statistical computing |
| [25] | R Development Core Team, (2009). R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria, ISBN 3-900051-07-0. URL: http://www.R-project.org; R Development Core Team, (2009). R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria, ISBN 3-900051-07-0. URL: http://www.R-project.org |
| [26] | Raftery, A. E.; Satagopan, J. M.; Newton, M. A.; Krivitsky, P. N., Estimating the integrated likelihood via posterior simulation using the harmonic mean identity, (Bayesian statistics 8 (2007), Oxford University Press: Oxford University Press Alicante, Spain), 1-45 · Zbl 1252.62038 |
| [27] | Roberts, G., Markov chain concepts related to sampling algorithms, (Gilks, W.; Richardson, S.; Spiegelhalter, D., Markov chain Monte Carlo in practice (1996), Chapman and Hall: Chapman and Hall London) · Zbl 0839.62078 |
| [28] | Rouder, J. N.; Lu, J., An introduction to Bayesian hierarchical models with an application in the theory of signal detection, Psychonomic Bulletin and Review, 12, 573-604 (2005) |
| [29] | Rouder, J. N.; Speckman, P. L.; Sun, D.; Morey, R. D.; Iverson, G., Bayesian \(t\)-tests for accepting and rejecting the null hypothesis, Psychonomic Bulletin and Review, 16, 225-237 (2009) |
| [30] | Scott, D. W., Multivariate density estimation (1992), John Wiley & Sons: John Wiley & Sons New York |
| [31] | Sellke, T.; Bayarri, M. J.; Berger, J. O., Calibration of \(p\) values for testing precise null hypotheses, American Statistician, 55, 62-71 (2001), URL: http://www.jstor.org/stable/2685531 · Zbl 1182.62053 |
| [32] | Stone, C. J.; Hansen, M. H.; Kooperberg, C.; Truong, Y. K., Polynomial splines and their tensor products in extended linear modeling, The Annals of Statistics, 25, 4, 1371-1425 (1997), URL: http://www.jstor.org/stable/2959054 · Zbl 0924.62036 |
| [33] | Tanner, M. A., Tools for statistical inference: methods for the exploration of posterior distributions and likelihood functions (1998), Springer: Springer New York |
| [34] | Verdinelli, I.; Wasserman, L., Computing Bayes factors using a generalization of the Savage-Dickey density ratio, Journal of the American Statistical Association, 90, 430, 614-618 (1995), URL: http://www.jstor.org/stable/2291073 · Zbl 0826.62022 |
| [35] | Wagenmakers, E.-J.; Lodewyckx, T.; Kuriyal, H.; Grasman, R., Bayesian hypothesis testing for psychologists: a tutorial on the SavageDickey method, Cognitive Psychology, 60, 158-189 (2010) |
| [36] | Wetzels, R.; Raaijmakers, J. G.W.; Jakab, E.; Wagenmakers, E.-J., How to quantify support for and against the null hypothesis: a flexible WinBUGS implementation of a default Bayesian ttest, Psychonomic Bulletin & Review, 16, 752-760 (2009) |
| [37] | Wetzels, Ruud; Grasman, Raoul P. P.P.; Wagenmakers, Eric-Jan, An encompassing prior generalization of the Savage-Dickey density ratio, Computational Statistics and Data Analysis, 54, 2094-2102 (2010) · Zbl 1284.62135 |
| [38] | Zellner, A.; Siow, A., Posterior odds ratios for selected regression hypotheses, (Bernardo, J. M.; DeGroot, M. H.; Lindley, D. V.; Smith, A. F.M., Bayesian statistics: proceedings of the first international meeting held in Valencia (Spain) (1980), University of Valencia), 585-603 · Zbl 0457.62004 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
