[R-meta] To include or not include time in rma.mv
Dear James, Thank you very much for the clarification. I'm assuming that you guys excluded the pretreatment effects mainly because your study pool consisted only of true experiments (right?). I might be wrong, but random assignment in single experiments could still suffer from bad luck and can equalize the groups only on average given innumerable replications. So, do you advise including a "time" interaction overall? Thank you so much for your time. Yuhang
On Tue, Sep 6, 2022 at 8:04 AM James Pustejovsky <jepusto at gmail.com> wrote:
Hi Yuhang, As one of the authors of the second study you mentioned, I think you may be misinterpreting our analysis. My understanding is that the data that we used to develop our examples did not include standardized mean differences prior to treatment. All of the included effect sizes were for post-treatment effects (i.e., `postwks >= 0`). I can't say for certain, but I suspect that the same thing holds for the Williams et al. meta-analysis (the first study you mentioned). If you look at their Table 2, the "Outcome timing" variable ranges from "midstream during intervention" to "follow-up post-test," so there don't appear to be any pre-treatment effect sizes included in the analysis. If the data did include pre-test effect sizes, then I fully agree that m1 would blur the interpretation of the treatment effects. James On Mon, Sep 5, 2022 at 7:36 PM Yuhang Hu <yh342 at nau.edu> wrote:
Hello All, I'm considering two candidate fixed-effects structures (cat_mod = categorical moderator) to meta-analyze a group of longitudinal studies (pre-test and follow-ups): m1 = rma.mv(yi ~ 0 + cat_mod + covariates, random = ~ 1 | study/effect) m2 = rma.mv(yi ~ 0 + cat_mod*time + covariates, random = ~ 1 | study/effect) In several authoritative meta-analyses of longitudinal studies such as ***https://doi.org/10.1080/19345747.2021.2009072*** and ***https://doi.org/10.1007/s11121-021-01246-3***, I noticed the authors used "m1" over "m2". But I was wondering, wouldn't, ignoring "time" (as in "m1") in the fixed-effects structure mix up the pre-test and follow-up effect sizes together which can blur our understanding of the treatment effect? Your guidance is highly appreciated, Yuhang [[alternative HTML version deleted]]
_______________________________________________ R-sig-meta-analysis mailing list @ R-sig-meta-analysis at r-project.org To manage your subscription to this mailing list, go to: https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
Yuhang Hu (She/Her/Hers) Ph.D. Student in Applied Linguistics Department of English Northern Arizona University [[alternative HTML version deleted]]