Message-ID: <CA+4peqGFpGwLmOKMoC8_pbXWS9XGqY_x0n4jdM41QRGcyO7tNw@mail.gmail.com>
Date: 2023-07-25T19:03:03Z
From: Nevo Sagi
Subject: [R-meta] Questions regarding REML and FE models and R^2 calculation in metafor
In-Reply-To: <CAFUVuJw-6F8GA8HgX3NAV_tSqgzYzrOEDRa5ukmw0vZoKMgtDQ@mail.gmail.com>
Thanks for the detailed explanation.
The link you included doesn't work for me. Is there another way to get to
that source?
?????? ??? ??, 25 ????? 2023, 21:37, ??? James Pustejovsky ?<
jepusto at gmail.com>:
> Hi Nevo,
>
> Responses inline below.
>
> Kind Regards,
> James
>
> On Tue, Jul 25, 2023 at 1:37?AM Nevo Sagi <nevosagi8 at gmail.com> wrote:
>
>> I don't understand the rationale of using random effects at the
>> experiment level. Experiments in my meta-analysis are parallel to
>> observations in a conventional statistical analysis.
>>
>
> I think this analogy doesn't follow. Conventional statistical analysis
> does have observation-level error terms (i.e., level-1 error)--it's just
> included by default as part of the model. In meta-analytic models, these
> errors are not included unless explicitly specified.
>
>
>> What is the meaning of using random effects at the observation level?
>>
>
> Observation-level random effects here are used to capture heterogeneity of
> effects across the experiments nested within a study. Considering that
> you're interested in looking at moderators that vary across the experiments
> reported in the same reference, it seems useful to attend to heterogeneity
> at this level as well.
>
>
>> In my understanding, by using random effects at the Reference level, I
>> already tell the model to look at within-reference variation.
>>
>
> This is not correct. Including reference-level random effects captures
> _between-reference_ variation (or heterogeneity) of effects.
>
>
>> In fact, the reason I was thinking to omit the random effect is because
>> the model was over-sensitive to variation in effect size across moderator
>> levels within specific references, while I am more interested in the total
>> variation across the whole moderator spectrum, and therefore I want to
>> focus more on the between-reference variation.
>> Does that make sense?
>>
>
> I stand by my original recommendation to consider including
> experiment-level heterogeneity here. Omitting the experiment-level
> heterogeneity more-or-less corresponds to averaging the effect size
> estimates together so that you have one effect per reference, which will
> tend to conceal within-reference heterogeneity. In fact, if you are using a
> model that does not include moderators / predictors that vary at the
> experiment level (within reference), then the correspondence is exact.
> Further details here: https://osf.io/preprints/metaarxiv/pw54r/
>
[[alternative HTML version deleted]]