[R-meta] Question about Cohen's d and meta-regression

Hi Angeline,

I don't quite understand what your concern is. Neither approach (of
computing a standardized mean change) is right or wrong -- they are just
different ways of standardizing the mean change. Also, a large treatment
effect does not automatically translate into a small correlation. If the
effect is such that the mean goes up (or down) by a large constant for
everybody, then the correlation will be 1. It is only when there is large
variability in the size of the treatment effect that the correlation will
be small. But again, I am not able to follow what you are concerned about,
so I cannot really answer your question.

Best,
Wolfgang

-----Original Message-----
From: Angeline Tsui [mailto:angelinetsui at gmail.com]
Sent: Monday, 08 January, 2018 0:15
To: Viechtbauer Wolfgang (SP)
Cc: r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Question about Cohen's d and meta-regression

Hello Wolfgang,

Thank you for your reply. Just a quick follow up question: I think that we
need to take into account of the correlation between pre-test and post-test
when we need to calculate the SD of difference score. I see this may be
important for my study because if there is a large treatment effect at
post-test, the correlation between pre-test and post-test would likely to
be small. In this case, it looks like that I should not use pre-test SD for
the SD of difference score. In this case, what is your recommendation?
Should I use post-test SD instead if SD of difference score is not provided?

Thanks,
Angeline

On Sun, Jan 7, 2018 at 5:24 PM, Viechtbauer Wolfgang (SP) <
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
A "within-subject Cohen's d" can be computed in two ways:

1) by dividing the mean difference by the SD of the change scores
(standardized mean change using change score standardization) or

2) by dividing the mean difference by the SD of the raw scores
(standardized mean change using raw score standardization). Here, the
pre-test SD is often used (although one could also use the post-test SD or
pool pre- and post-test SDs, although for the latter one needs to do some
extra work to get a correct estimate of the sampling variance).

The second approach yields a d value that is conceptually the same as a
"between-subject Cohen's d" (if we 'equate' the pre condition with the
control condition and the post condition with the experimental/treatment
condition and assume that there are no time effects). So, I see no need for
changing a within-subject Cohen's d to between-subject Cohen's d (leaving
aside how that would actually be done).

Best,
Wolfgang

-----Original Message-----
From: Angeline Tsui [mailto:angelinetsui at gmail.com]
Sent: Thursday, 04 January, 2018 13:11
To: Philipp Doebler; Viechtbauer Wolfgang (SP)
Cc: r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Question about Cohen's d and meta-regression

Dear Wolfgang and Philipp,

Thank you so much for the detailed response. This is exactly what I am
looking for. I will look into the Knapp & Hartung correction and also do a
power analysis of meta-regression. Thanks again for the recommended
resources.

Re Wolfgang as for the question about changing within-subject Cohen's d to
between-subject Cohen's d: To be honest, I have struggled quite a lot
before making this decision. The primary reason is due to recommendations
from researchers in my field. They once advocated meta-analysis in my field
to change within-subject Cohen's d to between-subject Cohen's d. I think it
is simply related to converting meta-analysis mean effect size to a common
metric, so that we can directly compare mean effect sizes across
meta-analyses. Another reason (I think I may be wrong and please feel free
to correct me) is that I was worried if I can still apply the Cohen's rule
of thumb (0.2, 0.5, 0.8) to interpret within-subject Cohen's d. I know that
this rule of thumb applies to between-subject Cohen's d, but I really am
not sure if we can do this for within-subject Cohen's d. Specifically, the
standard deviations are different between within-subject and
between-subject designs. The standard deviations for within-subject design
are theoretically smaller since we are controlling for individual
differences. Thus, I changed my metric to within-subject Cohen's d just for
the ease of interpretation as well.

Best,
Angeline