Skip to content

[R-meta] Calculating effect sizes from standardized regression coefficients in Metafor

4 messages · Wolf, Katrin, Wolfgang Viechtbauer

Wolf, Katrin
# 
Dear colleagues,

I am currently struggling with dealing with standardized regression coefficients (as indicator of the relationship between two variables of interest) in my meta-analysis with Metafor. Due to literature research, standardized regression coefficients can be used for meta-analysis when corresponding standard errors are also taken into account. Due to Metafor manual from 2023, it is possible to calculate effect size from partial correlations under consideration of t-statistics, sample size, number of predictors in regression model and R?. Do I interpret correctly that this is another approach?
I am sure there is a lot of experience with handling beta weights in Metafor. I would appreciate any information on this topic.

Kind regards,
Katrin

---
Dr. Katrin Wolf, Dipl.-Psych.
Wissenschaftliche Mitarbeiterin

Otto-Friedrich-Universit?t Bamberg
Lehrstuhl Fr?hkindliche Bildung und Erziehung
96045 Bamberg
1 day later
Wolfgang Viechtbauer
# 
Dear Katrin,

I am not sure I fully understand your question. I think you are referring to escalc() with measure "PCOR", which calculates partial correlation coefficients (from things like the corresponding t-statistics of the regression coefficients), but your phrasing (that this "calculates effect size from partial correlations") is confusing me.

If you want to meta-analyze standardized regression coefficients and have the corresponding standard errors, then one can of course also meta-analyze those directly. However, note that the standard errors of standardized regression coefficients are typically not computed in the most accurate way (i.e., the standard errors one obtains by fitting a regression model to standardized variables ignore that the variances used to standardize the variables are estimated). See, for example:

Jones, J. A., & Waller, N. G. (2013). Computing confidence intervals for standardized regression coefficients. Psychological Methods, 18(4), 435-453. https://doi.org/10.1037/a0033269

If you have the full correlation matrix of the variables in each regression model, one can compute more appropriate standard errors, but this is unlikely to be the case in practice.

Best,
Wolfgang
Wolf, Katrin
# 
Dear Wolfgang,

Thank you very much for your response! Please let me summarize to check if I understood correctly. As I could also see from the correspondence with Rasheda, there are two possible ways for dealing with regression coefficients:
1. Using escalc with "PCOR" (under consideration of t-statistics, sample size, number of predictors and variance explained)
2. directly meta-analyze beta coefficients as effect size (yi) in one of the rma-commands. In this case vi would be variance of beta coefficient meaning square of SE? 
    (I am going to check the paper you mentioned regarding the standard errors.)

Is one of these approaches better than the other - considering the fact that I aim at aggregating effect sizes from different kind of measures: regression coefficients, correlation coefficients (Fisher?s r-to-z transformed correlation coefficient) and group mean differences (standardized mean difference)?

Best,
Katrin


-----Urspr?ngliche Nachricht-----
Von: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer at maastrichtuniversity.nl> 
Gesendet: Mittwoch, 12. Juli 2023 16:57
An: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis at r-project.org>
Cc: Wolf, Katrin <katrin.wolf at uni-bamberg.de>
Betreff: RE: Calculating effect sizes from standardized regression coefficients in Metafor

Dear Katrin,

I am not sure I fully understand your question. I think you are referring to escalc() with measure "PCOR", which calculates partial correlation coefficients (from things like the corresponding t-statistics of the regression coefficients), but your phrasing (that this "calculates effect size from partial correlations") is confusing me.

If you want to meta-analyze standardized regression coefficients and have the corresponding standard errors, then one can of course also meta-analyze those directly. However, note that the standard errors of standardized regression coefficients are typically not computed in the most accurate way (i.e., the standard errors one obtains by fitting a regression model to standardized variables ignore that the variances used to standardize the variables are estimated). See, for example:

Jones, J. A., & Waller, N. G. (2013). Computing confidence intervals for standardized regression coefficients. Psychological Methods, 18(4), 435-453. https://doi.org/10.1037/a0033269

If you have the full correlation matrix of the variables in each regression model, one can compute more appropriate standard errors, but this is unlikely to be the case in practice.

Best,
Wolfgang
Wolfgang Viechtbauer
# 
Hi Katrin,

For "PCOR" (partial correlations), you need to know the t-statistic (or the corresponding p-value), sample size, and the number of predictors (unless you already know the partial correlation directly, but this is quite rare). For "SPCOR", you also need the R^2 from the model.

If you go that route, then I would recommend using "ZPCOR" for the Fisher r-to-z transformed partial correlation coefficients. So yes, this is option 1.

Another option - if you already have the standardized regression coefficients and corresponding standard errors - is to directly meta-analyze those.

It is difficult to say which option is better in general, but option 1 does not suffer from the problem that I explained in my other post (about the standard errors of standardized regression coefficients -- although to what extent this is really a problem is context specific).

Best,
Wolfgang