[R-meta] calculate effect size for pre-post control design

Yang,

As you noted, the difference between the two effect size metrics is in the
denominator of the standardized mean difference. Standardizing by the
change-score SD is what it sounds like: the denominator is the standard
deviation of the (Post - Pre) change scores. If the SD of the pre-test
scores is equal to the SD of the post-test scores, then SD(Post - Pre) =
SD(Pre) * sqrt(2 * (1 - r)), where r = corr(Post, Pre). So d_prepost =
d_pre / sqrt(2 * (1 - r)) and d_prepost will be larger than d_pre (or
d_post) whenever r > 0.5.

Whether to use change score standardization or pretest/post-test
standardization depends on the characteristics of the studies you are
trying to synthesize. Suppose that your pool of studies includes some that
use pre-post designs and some that use post-test only designs. Change score
standardization is not viable here because there's no way to compute change
scores from the post-test only designs, but post-test standardization will
work so long as the post-test SDs can be obtained for all of the studies.
In syntheses I've worked on, we've used pre-test standardization for
pre-post designs and post-test standardization for studies where pre-test
data is not available, on the assumption that the pre-test and post-test
SDs will usually be similar.

Another consideration is measurement error. If the studies that are
included in your synthesis use different outcome scales with varying
degrees of reliability, then change score standardization will be less
defensible because taking change scores amplifies the effects of
measurement error. In comparison, differential reliability will have a
weaker effect on effect sizes based on pre-test standardization.

One other note: your r seems unrealistically high--was that a typo? If not,
I would recommend checking through the study again to see if there might be
some error.

James

On Mon, Jan 21, 2019 at 7:37 PM Yingkai Yang <yykjiyisuipian at gmail.com>
wrote:

[R-meta] calculate effect size for pre-post control design

Thread (3 messages)