[R-meta] Correcting gain effects in nested studies
Ah, sorry I missed that you want SMCC rather than SMCR. Taylor and WWC focus on SMCR. I'm not aware of anything developed specifically for SMCC. Apart from the regular critiques of SMCC, there's also the difficulty of determining an appropriate ICC for the change scores. Just speculating, but I would guess that ICCs for change scores might be quite different---and probably smaller?---than ICCs for outcomes measured cross-sectionally (for which there are many sources of empirical data available). James
On Wed, May 8, 2024 at 8:58?AM Zhouhan Jin <zjin65 at uwo.ca> wrote:
Thank you, James! I might be missing something, but I couldn't specifically locate the formulas to adjust SMCC (and its SE) in ?metafor::escalc, certainly in Taylor et al. Regarding WWC, is there a specific equation that allows for nestedness-adjusting of SMCC (defined below)? SMCC = (postM - preM) / SD_of_change_scores with its SE in metafor by default (vtype='LS') being calculated without a small-sample correction as: SE_SMCC = sqrt( (1/n) + ((SMCC^2) / (2*n)) ) Many thanks! Best wishes, Zhouhan On May 7, 2024 at 21:30 -0400, James Pustejovsky <jepusto at gmail.com>, wrote: See Taylor, Pigott, and Williams (2022; https://doi.org/10.3102/0013189X211051319) for how to handle cluster-randomized trials that involve gain scores or covariate adjustment. They provide a shiny app too. The technical details are also described in Appendix E of the What Works Clearinghouse handbook (Version 5; https://ies.ed.gov/ncee/WWC/Docs/referenceresources/Final_WWC-HandbookVer5_0-0-508.pdf). See pp. 173-174 The methods described in these sources are consistent with the "general recipe" for standardized mean difference estimates as described here: https://www.jepusto.com/alternative-formulas-for-the-smd/ James On Tue, May 7, 2024 at 7:14?PM Zhouhan Jin <zjin65 at uwo.ca> wrote:
Hello All,
Hedges (2007) provides formulas for adjusting SMD effects (g) and their
SEs for when primary studies have a nested design (below).
But I want to compute gain effects (ex. SMCC in metafor::escalc) from my
nested studies, not SMDs.
So, how can I adjust my SMCCs and their SEs for nestedness in the primary
studies?
adjusted_g = g * sqrt(1 - ((2 * (n_bar - 1) * icc) /
(n_cluster * n_bar - 2)))
adjusted_SE = ((Nt+Nc)/(Nt*Nc))*(1 + ((n_bar- 1)*icc)) +
( g^2 * (
(((N_tot -2)*(1-icc)^2 ) + (n_bar*(N_tot - 2*n_bar)*icc^2) +
(2* (N_tot - 2*n_bar) * icc * (1 - icc)) ) /
((2* (N_tot-2)) * ( (N_tot-2) - (2* (n_bar-1)*icc) ))
) )
Thanks a lot!
Best wishes,
Zhouhan