[R-meta] SMD from three-level nested design (raw data available)

Dear James,

Thanks so much for your reply, this is really helpful and made me think
carefully about the data I'm dealing with. The effect I'm trying to compute
is defined by Hedges (2009, p. 348) as d_BC, i.e. the treatment effect at
level 2 of a 3-level design. In "my" dataset, the unit of measurement is
the allocation decision (level 1), and the unit of randomization is the
group (level 3). The effect I'm after, however, is the treatment effect at
the level of the participant (level 2).

Unfortunately, Hedges (2009) does not provide the equation for the
computation of d_BC using fixed-effect estimates and variance components.
However, in the context of a 2-level model, Hedges (2009) defines the
between-cluster effect as

d_B = b / sig_B  [Eq. 18.17]

where b is the estimated fixed effect and sig_B^2 is the between-cluster
variance component. Note that the within-cluster variance component is
omitted from the denominator. By contrast, the total treatment effect is
defined as

d_T = b / sqrt(sig_B^2 + sig_W^2)  [Eq. 18.23]

where b is again the estimated fixed effect, sig_B^2 is the between-cluster
variance component, and sig_W^2 is the within-cluster variance component. I
tried to apply this logic to the study I'm coding, in which the effect size
of interest is not the total treatment effect, but rather the treatment
effect at the level of individual participants (level 2). As such, I
omitted sig_w from the denominator. My understanding is that if I add the
repeated-measures variance component to the denominator, as you suggested,
I would get the treatment effect at the level of the allocation decision
(as per Hedges, 2009, Eq. 18.55). And wouldn't such an effect size be
incomparable to the other SMDs in the meta-analysis, which represent a
treatment effect at the level of participants?

Many thanks for your help,
Fabian

---
Reference:
Hedges, L. V. (2009). Effect sizes in nested designs. In Cooper, H.,
Hedges, L. V., & Valentine, J. C. (Eds.), The Handbook of Research
Synthesis and Meta-Analysis (pp. 337-355). New York: Russell Sage
Foundation.

[R-meta] SMD from three-level nested design (raw data available)

Thread (7 messages)