[R-meta] [External] RE: Effect Sizes and Beta Coefficients
Dear Rebecca In general estimates adjusted for different sets of covariates are not directly comparable. If the sets are almost the same then you may be able to argue for including them. It might also depend on strategic questions like how many studies you already have and whether adding one more is worth the hassle of explaining why you included it. In short as so often the answer is, how brave do you feel? Michael
On 17/06/2024 15:27, Hall, Rebecca via R-sig-meta-analysis wrote:
Dear Wolfgang, Many thanks for your assistance so far. We are still looking at papers using various statistical analyses, and have come across one which utilises a one-way MANCOVA. Please could you advise on the validity of calculating an effect size from this type of analysis, as the reported F and p values are altered by controlling for covariates, whereas this has not been the case in other studies. Similarly, we have a paper reporting odds ratios which, if we are correct, we can directly convert to an effect size (pearson's r). However, the odds ratios in question are presented as two models, one with more covariates than the other. Is there a particular model you would suggest including (see below)? Best, Rebecca *Dr Rebecca Hall | Research Associate * Department of Psychology | Lancaster University ------------------------------------------------------------------------ *From:* Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer at maastrichtuniversity.nl> *Sent:* 13 June 2024 13:43 *To:* Hall, Rebecca <r.hall5 at lancaster.ac.uk>; R Special Interest Group for Meta-Analysis <r-sig-meta-analysis at r-project.org> *Subject:* RE: [External] RE: [R-meta] Effect Sizes and Beta Coefficients Dear Rebecca, Please see below for my answer. Best, Wolfgang
-----Original Message----- From: Hall, Rebecca <r.hall5 at lancaster.ac.uk> Sent: Wednesday, June 12, 2024 17:56 To: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer at maastrichtuniversity.nl>; R Special Interest Group for Meta-Analysis <r-sig-meta-analysis at r-project.org> Subject: Re: [External] RE: [R-meta] Effect Sizes and Beta Coefficients Dear Wolfgang, Many thanks for your response. Am I correct in thinking that I can convert a p- value in R using qt(1- p-value, n-1)?
I assume you are asking about a p-value from a regression model. First of all, this is typically a two-sided p-value, so the given p-value must first be divided by 2. Also, the degrees of freedom are n-p-1 for a regression model with p regression coefficients. So it would be: qt(1-pval/2, df=n-p-1) Note that the (two-sided) p-value does not tell you anything about the sign of the regression coefficient (which in turn corresponds to the sign of the test statistic and hence the sign of the (semi)partial correlation), so additional information reported needs to be used to determine the correct sign.
Further to this, please could you advise how I might then calculate the partial correlation from a one-sample t-test?
You can't. A one-sample t-test is used to test if the mean of a single variable differs significantly from some value. It is not possible to compute the partial correlation for a single variable.
Best, Rebecca Dr Rebecca Hall | Research Associate Department of Psychology | Lancaster University
________________________________________ From: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer at maastrichtuniversity.nl> Sent: 11 June 2024 17:21 To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis at r- project.org> Cc: Hall, Rebecca <r.hall5 at lancaster.ac.uk> Subject: [External] RE: [R-meta] Effect Sizes and Beta Coefficients This email originated outside the University. Check before clicking links or attachments. Dear Rebecca, This is my personal opinion: I would consider this approach outdated. Typically, in a situation like this, one also knows the t-statistic for the coefficient of interest (or its p-value from which one can back-calculate the t- statistic). In that case, one can compute the (semi)partial correlation coefficient for the coefficient. However, whether one should combine such 'partial' effect sizes with bivariate correlations is debatable in the first place. A relevant article that essentially argues against this is: Aloe, A. M. (2015). Inaccuracy of regression results in replacing bivariate correlations. Research Synthesis Methods, 6(1), 21-27. https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1002%2Fjrsm.1126&data=05%7C02%7Challr4%40live.lancs.ac.uk%7C2ce13474f1ec470a8a9c08dc8ba66749%7C9c9bcd11977a4e9ca9a0bc734090164a%7C0%7C0%7C638538794019673955%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=OUXIjY07mR54gO5eHMgodzvyAuc3uB4CnWfESXhYNJA%3D&reserved=0 <https://doi.org/10.1002/jrsm.1126> At least, one could try to capture some of the heterogeneity introduced by this by including a moderator in the model that indicates the type of correlation coefficient. With enough studies, one could even go a step further and include moderators that indicate which covariates were included in the original regression models from which the (semi)partial correlations were obtained (as a bunch of dummy variables). Best, Wolfgang -----Original Message----- From: R-sig-meta-analysis <r-sig-meta-analysis-bounces at r-project.org> On Behalf Of Hall, Rebecca via R-sig-meta-analysis Sent: Tuesday, June 11, 2024 17:57 To: r-sig-meta-analysis at r-project.org Cc: Hall, Rebecca <r.hall5 at lancaster.ac.uk> Subject: [R-meta] Effect Sizes and Beta Coefficients Dear all, I have a question regarding the use of a beta coefficient as a substitute for effect size where a paper lacks statistical data for Pearson's r to otherwise be calculated. Peterson & Brown (2005) support the use of standardised beta coefficients and relative SE in the place of correlations, but Roth et al. (2018) criticise this. I'm therefore wondering whether there is a general consensus regarding the use of beta coefficients, and should Peterson & Brown's approach no longer be appropriate then I would be gladly advised on the alternative method that should be utilised. Many thanks, Rebecca Dr Rebecca Hall | Research Associate Department of Psychology | Lancaster University
<http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> Virus-free.www.avg.com <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
_______________________________________________ R-sig-meta-analysis mailing list @ R-sig-meta-analysis at r-project.org To manage your subscription to this mailing list, go to: https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
Michael