×

Fisher transformation based confidence intervals of correlations in fixed- and random-effects meta-analysis. (English) Zbl 1534.62246

Summary: Meta-analyses of correlation coefficients are an important technique to integrate results from many cross-sectional and longitudinal research designs. Uncertainty in pooled estimates is typically assessed with the help of confidence intervals, which can double as hypothesis tests for two-sided hypotheses about the underlying correlation. A standard approach to construct confidence intervals for the main effect is the Hedges-Olkin-Vevea Fisher-z (HOVz) approach, which is based on the Fisher-z transformation. Results from previous studies [A. P. Field, Psychological Methods 10, No. 4, 444–467 (2005; doi:10.1037/1082-989X.10.4.444); A. R. Hafdahl and M. A. Williams, ibid. 14, No. 1, 24–42 (2009; doi:10.1037/a001469]), however, indicate that in random-effects models the performance of the HOVz confidence interval can be unsatisfactory. To this end, we propose improvements of the HOVz approach, which are based on enhanced variance estimators for the main effect estimate. In order to study the coverage of the new confidence intervals in both fixed- and random-effects meta-analysis models, we perform an extensive simulation study, comparing them to established approaches. Data were generated via a truncated normal and beta distribution model. The results show that our newly proposed confidence intervals based on a Knapp-Hartung-type variance estimator or robust heteroscedasticity consistent sandwich estimators in combination with the integral z-to-r transformation [A. R. Hafdahl, Br. J. Math. Stat. Psychol. 62, No. 2, 233–261 (2009; doi:10.1348/000711008X281633)] provide more accurate coverage than existing approaches in most scenarios, especially in the more appropriate beta distribution simulation model.
© 2021 The Authors. British Journal of Mathematical and Statistical Psychology published by John Wiley & Sons Ltd on behalf of British Psychological Society

MSC:

62P15 Applications of statistics to psychology

Software:

metafor

References:

[1] Agresti, A., & Coull, B. A. (1998). Approximate is better than “Exact” for interval estimation of binomial proportions. The American Statistician, 52, 119-126. https://doi.org/10.1080/00031305.1998.10480550 · doi:10.1080/00031305.1998.10480550
[2] Aldao, A., Nolen‐Hoeksema, S., & Schweizer, S. (2010). Emotion‐regulation strategies across psychopathology: A meta‐analytic review. Clinical Psychology Review, 30, 217-237. https://doi.org/10.1016/j.cpr.2009.11.004 · doi:10.1016/j.cpr.2009.11.004
[3] Barrick, M. R., & Mount, M. K. (1991). The big five personality dimensions and job performance: a meta‐analysis. Personnel Psychology, 44, 1-26. https://doi.org/10.1111/j.1744‐6570.1991.tb00688.x · doi:10.1111/j.1744‐6570.1991.tb00688.x
[4] Chalkidou, A., Landau, D. B., Odell, W., Cornelius, V. R., O’Doherty, M. J., & Marsden, P. K. (2012). Correlation between Ki‐67 immunohistochemistry and 18F‐fluorothymidine uptake in patients with cancer: a systematic review and meta‐analysis. European journal of cancer, 48(18), 3499-3513. https://doi.org/10.1016/j.ejca.2012.05.001 · doi:10.1016/j.ejca.2012.05.001
[5] Cheung, M.W.L. (2015). Meta‐analysis: A structural equation modeling approach. Hoboken, NJ: John Wiley & Sons. · Zbl 1311.62002
[6] Costa, Jr, P. T., & McCrae, R. R. (1985). The NEO personality inventory. Lutz, FL: Psychological Assessment Resources Odessa. https://www.parinc.com/PAR_Support
[7] Costa, Jr, P. T., & McCrae, R. R. (2008). The revised NEO personality inventory (NEO‐PI‐R). Thousand Oaks, CA: Sage Publications. https://us.sagepub.com/en‐us/nam/contact‐us
[8] Cribari‐Neto, F., Souza, T. C., & Vasconcellos, K. L. (2007). Inference under heteroskedasticity and leveraged data. Communication in Statistics - Theory and Methods, 36, 1877-1888. https://doi.org/10.1080/03610920601126589 · Zbl 1315.62059 · doi:10.1080/03610920601126589
[9] Cribari‐Neto, F., & Zarkos, S. G. (2004). Leverage‐adjusted heteroskedastic bootstrap methods. Journal of Statistical Computation and Simulation, 74, 215-232. https://doi.org/10.1080/0094965031000115411 · Zbl 1048.62063 · doi:10.1080/0094965031000115411
[10] Field, A. P. (2001). Meta‐analysis of correlation coefficients: A Monte Carlo comparison of fixed‐and random‐effects methods. Psychological Methods, 6, 161-180. https://doi.org/10.1037/1082‐989X.6.2.161 · doi:10.1037/1082‐989X.6.2.161
[11] Field, A. P. (2005). Is the meta‐analysis of correlation coefficients accurate when population correlations vary?Psychological Methods, 10, 444-467. https://doi.org/10.1037/1082‐989X.10.4.444 · doi:10.1037/1082‐989X.10.4.444
[12] Fisher, R. A. (1915). Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population. Biometrika, 10, 507-521. https://doi.org/10.2307/2331838 · doi:10.2307/2331838
[13] Hafdahl, A. R. (2009). Improved Fisher z estimators for univariate random‐effects meta‐analysis of correlations. British Journal of Mathematical and Statistical Psychology, 62, 233-261. https://doi.org/10.1348/000711008X281633 · doi:10.1348/000711008X281633
[14] Hafdahl, A. R., & Williams, M. A. (2009). Meta‐analysis of correlations revisited: Attempted replication and extension of Field’s (2001) simulation studies. Psychological Methods, 14, 24-42. https://doi.org/10.1037/a0014697 · doi:10.1037/a0014697
[15] Hartung, J. (1999). An alternative method for meta‐analysis. Biometrical Journal: Journal of Mathematical Methods in Biosciences, 41, 901-916. https://doi.org/10.1002/(SICI)1521‐4036(199912)41:8<901::AID‐BIMJ901>3.0.CO;2‐W · Zbl 0944.62103 · doi:10.1002/(SICI)1521‐4036(199912)41:8<901::AID‐BIMJ901>3.0.CO;2‐W
[16] Hartung, J., & Knapp, G. (2001). A refined method for the meta‐analysis of controlled clinical trials with binary outcome. Statistics in Medicine, 20, 3875-3889. https://doi.org/10.1002/sim.1009 · doi:10.1002/sim.1009
[17] Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta‐analysis. San Diego, CA: Academic Press. · Zbl 0666.62002
[18] Hedges, L., & Vevea, J. (1998). Fixed‐and random‐effects models in meta‐analysis. Psychological Methods, 3, 486-504.
[19] Hotelling, H. (1953). New light on the correlation coefficient and its transforms. Journal of the Royal Statistical Society, Series B, 15, 193-232. https://doi.org/10.1111/j.2517‐6161.1953.tb00135.x · Zbl 0052.14905 · doi:10.1111/j.2517‐6161.1953.tb00135.x
[20] Hunter, J. E., & Schmidt, F. L. (1994). Estimation of sampling error variance in the meta‐analysis of correlations: Use of average correlation in the homogeneous case. Journal of Applied Psychology, 79, 171.
[21] Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta‐analysis: Correcting error and bias in research findings. Thousand Oaks, CA: Sage Publishing.
[22] IntHout, J., Ioannidis, J. P., & Borm, G. F. (2014). The Hartung‐Knapp‐Sidik‐Jonkman method for random effects meta‐analysis is straightforward and considerably outperforms the standard DerSimonian‐Laird method. BMC Medical Research Methodology, 14, 1-12. https://doi.org/10.1186/1471‐2288‐14‐25 · doi:10.1186/1471‐2288‐14‐25
[23] Jak, S. (2015). Meta‐analytic structural equation modelling. New York: Springer. · Zbl 1329.62010
[24] Johnson, N. L., Kotz, S., & Balakrishnan, N. (1994). Continuous univariate distributions. New York: John Wiley & Sons. · Zbl 0811.62001
[25] Langan, D., Higgins, J. P., Jackson, D., Bowden, J., Veroniki, A. A., Kontopantelis, E., … Simmonds, M. (2019). A comparison of heterogeneity variance estimators in simulated random‐effects meta‐analyses. Research Synthesis Methods, 10, 83-98. https://doi.org/10.1002/jrsm.1316 · doi:10.1002/jrsm.1316
[26] Lehmann, E. L. (2004). Elements of large‐sample theory. Springer Science and Business media, Luxemburg.
[27] Molloy, G. J., O’Carroll, R. E., & Ferguson, E. (2013). Conscientiousness and medication adherence: A meta‐analysis. Annals of Behavioral Medicine, 47, 92-101. https://doi.org/10.1007/s12160‐013‐9524‐4 · doi:10.1007/s12160‐013‐9524‐4
[28] Morris, T. P., White, I. R., & Crowther, M. J. (2019). Using simulation studies to evaluate statistical methods. Statistics in Medicine, 38, 2074-2102. https://doi.org/10.1002/sim.8086 · doi:10.1002/sim.8086
[29] Omelka, M., & Pauly, M. (2012). Testing equality of correlation coefficients in two populations via permutation methods. Journal of Statistical Planning and Inference, 142, 1396-1406. https://doi.org/10.1016/j.jspi.2011.12.018 · Zbl 1242.62036 · doi:10.1016/j.jspi.2011.12.018
[30] Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science, 349. https://doi.org/10.1126/science.aac4716 · doi:10.1126/science.aac4716
[31] Osburn, H., & Callender, J. (1992). A note on the sampling variance of the mean uncorrected correlation in meta‐analysis and validity generalization. Journal of Applied Psychology, 77, 115-122. https://doi.org/10.1037/0021‐9010.77.2.115 · doi:10.1037/0021‐9010.77.2.115
[32] Rosopa, P. J., Schaffer, M. M., & Schroeder, A. N. (2013). Managing heteroscedasticity in general linear models. Psychological Methods, 18, 335-351. https://doi.org/10.1037/a0032553 · doi:10.1037/a0032553
[33] Santos, S., Almeida, I., Oliveiros, B., & Castelo‐Branco, M. (2016). The role of the amygdala in facial trustworthiness processing: A systematic review and meta‐analyses of fMRI studies. PloS one, 11(11).e0167276. https://doi.org/10.1371/journal.pone.0167276 · doi:10.1371/journal.pone.0167276
[34] Schmid, F., & Schmidt, R. (2007). Multivariate extensions of Spearman’s rho and related statistics. Statistics & Probability Letters, 77, 407-416. https://doi.org/10.1016/j.spl.2006.08.007 · Zbl 1108.62056 · doi:10.1016/j.spl.2006.08.007
[35] Schober, P., Boer, C., & Schwarte, L. A. (2018). Correlation coefficients: Appropriate use and interpretation. Anesthesia & Analgesia, 126, 1763-1768. https://doi.org/10.1213/ANE.0000000000002864 · doi:10.1213/ANE.0000000000002864
[36] Schulze, R. (2004). Meta‐analysis: A comparison of approaches. Boston, MA: Hogrefe Publishing. https://www.hogrefe.com/us/contact
[37] Schwarzer, G., Carpenter, J. R., & Rücker, G. (2015). Meta‐analysis with R. Cham: Springer. · Zbl 1333.92002
[38] Sidik, K., & Jonkman, J. N. (2005). Simple heterogeneity variance estimation for meta‐analysis. Applied Statistics, 54, 367-384. https://doi.org/10.1111/j.1467‐9876.2005.00489.x · Zbl 1490.62377 · doi:10.1111/j.1467‐9876.2005.00489.x
[39] Sidik, K., & Jonkman, J. N. (2006). Robust variance estimation for random effects meta‐analysis. Computational Statistics & Data Analysis, 50, 3681-3701. https://doi.org/10.1016/j.csda.2005.07.019 · Zbl 1445.62287 · doi:10.1016/j.csda.2005.07.019
[40] Sirin, S. R. (2005). Socioeconomic status and academic achievement: A meta‐analytic review of research. Review of Educational Research, 75, 417-453. https://doi.org/10.3102/00346543075003417 · doi:10.3102/00346543075003417
[41] Süli, E., & Mayers, D. F. (2003). An introduction to numerical analysis. Cambridge, UK: Cambridge University Press. · Zbl 1033.65001
[42] Veroniki, A. A., Jackson, D., Viechtbauer, W., Bender, R., Bowden, J., Knapp, G., … Salanti, G. (2016). Methods to estimate the between‐study variance and its uncertainty in meta‐analysis. Research Synthesis Methods, 7, 55-79. https://doi.org/10.1002/jrsm.1164 · doi:10.1002/jrsm.1164
[43] Viechtbauer, W. (2010). Conducting meta‐analyses in R with the metafor package. Journal of Statistical Software, 36, 1-48. https://doi.org/10.18637/jss.v036.i03 · doi:10.18637/jss.v036.i03
[44] Viechtbauer, W., López‐López, J. A., Sánchez‐Meca, J., & Marn‐Martnez, F. (2015). A comparison of procedures to test for moderators in mixed‐effects meta‐regression models. Psychological Methods, 20, 360-374. https://doi.org/10.1037/met0000023 · doi:10.1037/met0000023
[45] Welz, T., & Pauly, M. (2020). A simulation study to compare robust tests for linear mixed‐effects meta‐regression. Research Synthesis Methods, 11, 331-342. https://doi.org/10.1002/jrsm.1388 · doi:10.1002/jrsm.1388
[46] White, H. (1980). A heteroskedasticity‐consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica, 48, 817-838. https://doi.org/10.2307/1912934 · Zbl 0459.62051 · doi:10.2307/1912934
[47] Wu, C.F. J. (1986). Jackknife, bootstrap and other resampling methods in regression analysis. Annals of Statistics, 14, 1261-1295. https://doi.org/10.1214/aos/1176350142 · Zbl 0618.62072 · doi:10.1214/aos/1176350142
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.