[R-meta] Meta-Analysis using different correlation coefficients
Dear Lukasz, Thank you so much for answering, we really appreciate the effort and your time. We had also done some research, which lead us to the following thread from Wolfgang Viechtbauer: https://chat.stackexchange.com/rooms/60238/discussion-between-mark-white-and-wolfgang <https://chat.stackexchange.com/rooms/60238/discussion-between-mark-white-and-wolfgang>, and further to his paper (Jacobs & Viechtbauer) regarding point-biserial and biserial correlations (?Estimation of the biserial correlation and its sampling variance for use in meta-analysis?) from 2016 (https://onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1218 <https://onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1218>). Based on this we were a little worried about including point-biserial and Pearson?s product-moment correlations in the same meta-analysis without transforming the point-biserial coefficients? E.g., in the paper they are saying ?Unlike the point-biserial correlation coefficient, biserial coefficients can therefore be integrated with product-moment correlation coefficients in the same meta-analysis?, which lead us to assume (contrary to your suggestion) that point-biserial and product-moment correlation coefficients should not be included in the same meta-analysis without further transformation? But the paper also exclusively treats cases where a variable was *artificially* dichotomized (and still analyzed with point-biserial correlation instead of biserial correlation). But you are suggesting that in case the point-biserial correlation is based on a naturally occurring binary variable (e.g., gender), we can integrate it with Pearson's poruduct-moment correlation in the same analysis? Thank you so much in advance and best wishes Lena
Am 15.02.2022 um 08:03 schrieb Lukasz Stasielowicz <lukasz.stasielowicz at uni-osnabrueck.de>: Dear Lena, since you haven't received a response yet, I will address some points: It is important to define a target effect size for the meta-analysis, e.g. Pearson correlation r. If other effect sizes are reported in the primary studies then one could transform them or use the raw data to compute the target effect size. If Pearson correlation is the target effect size and you have point-biserial correlations then no transformation is necessary because it is mathematically equivalent to Pearson correlation. Spearman correlation and Pearson correlation will not always lead to similar values, as the latter is less likely to detect nonlinear (monotonic) relationships. Therefore, transforming Spearman correlations to Pearson correlations could ensure, that the effect size values can be interpreted the same way. In this particular case one could use arcsin formulas: de Winter, J. C. F., Gosling, S. D., & Potter, J. (2016). Comparing the Pearson and Spearman correlation coefficients across distributions and sample sizes: A tutorial using simulations and empirical data. Psychological Methods, 21(3), 273?290. doi.org/10.1037/met0000079 You'll find formulas for other transformations in textbooks and other journal articles. While many meta-analysts tend to use all available information and apply all possible transformations (d --> r, Spearman --> Pearson, OR --> r), there are some people, who would point out that under certain circumstances not all transformations are valid (e.g., d --> r can the distinction between control group and experimental group be thought as a part of an underlying continuous distribution or is it a natural dichotomy?). However, some people would argue that every effect size is useful (in particular, if the meta-analysis is small). Sometimes meta-analysts conduct a moderator analysis and compare converted effect sizes (e.g., d-->r) with effect sizes that didn't require converting (e.g. r), in order to check the influence of pooling different designs/effects. Multiple effect sizes: If there are multiple studies with multiple effect sizes then a three-level meta-analysis could be useful. Alternatively, one could use cluster-robust variance estimation, e.g. cran.r-project.org/web/packages/clubSandwich/vignettes/meta-analysis-with-CRVE.html If there is only one sample with multiple effects then choosing one effect size (e.g., the typical operationalization of the constructs, the most valid operationalization) could be an option. Alternatively, one could compute a mean effect size, as you have mentioned in your message. I hope it answers at least some of your questions. Good luck with your project! Best, Lukasz -- Lukasz Stasielowicz Osnabr?ck University Institute for Psychology Research methods, psychological assessment, and evaluation Seminarstra?e 20 49074 Osnabr?ck (Germany) Am 09.02.2022 um 07:54 schrieb r-sig-meta-analysis-request at r-project.org:
Send R-sig-meta-analysis mailing list submissions to r-sig-meta-analysis at r-project.org To subscribe or unsubscribe via the World Wide Web, visit
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis or, via email, send a message with subject or body 'help' to r-sig-meta-analysis-request at r-project.org You can reach the person managing the list at r-sig-meta-analysis-owner at r-project.org When replying, please edit your Subject line so it is more specific than "Re: Contents of R-sig-meta-analysis digest..." Today's Topics: 1. Meta-Analysis using different correlation coefficients (Lena Pollerhoff) 2. Meta-Analysis using different correlation coefficients (Lena Pollerhoff) 3. Comparison to baseline- remove intercept or keep? (Danielle Hiam) 4. questions on some functions in metafor and clubsandwich (Brendan Hutchinson) ---------------------------------------------------------------------- Message: 1 Date: Tue, 8 Feb 2022 13:39:52 +0100 From: Lena Pollerhoff <lena at pollerhoff.de> To: r-sig-meta-analysis at r-project.org Subject: [R-meta] Meta-Analysis using different correlation coefficients Message-ID: <AA0B9D07-6B89-4BF7-91F7-D32B5FDE031A at pollerhoff.de> Content-Type: text/plain; charset="utf-8" Hello, We are currently conducting a meta-analysis based on correlation coefficients. We have received a huge amount of raw datasets, so that we are able to calculate effect sizes/correlations coefficients on our own for many datasets, and we have other correlations extracted from the original pubs. Therefore I have a couple of questions: 1. If one variable is dichotomous and the other variable is continuous but not normally distributed, what kind of coefficient should be calculated? We?d go for point-biserial if the variable is naturally dichotomous (not artificially dichotomized), and for biserial correlation if the dichotomous variable was artificially dichotomized, but are worried that both require normal distribution of the continuous variable? 2. We are wondering how to best integrate person?s product moment correlation coefficients (both continuous, normally distributed variables), (point-) biserial correlation coefficients (for 1 (artificial) dichotomous and 1 continuous variable) and spearman rang correlation coefficients (for non-parametric, both continuous variables) in one meta-analysis? Just use the raw values? Or is it better to transform them in a homogenous way (I?ve read Fisher?s z makes less sense for anything else than Pearson?s r as a variance-stabilizing procedure?)? Can spearman rho be converted using fisher?s z transformation? I?ve also read that it is not advisable to include product-moment correlation and point-biserial correlation in one meta-analysis, is there a way to convert the point-biserial correlation to something that can be integrated with Pearson?s r and Spearman?s rho? 3. I have multiple effect sizes within one sample and I want to aggregate them, how do I define rho in the aggregate function from the metafor package? Is it possible to calculate rho based on the raw datasets? Or would it better to think in a conservative way and assume perfect redundancy (i.e., rho = 0.9)? Thanks in advance for your time and effort! Best, Lena
_______________________________________________ R-sig-meta-analysis mailing list R-sig-meta-analysis at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis