[R-meta] Adding random effect to SCE models
Dear Fred, Just to clarify my answer in the post you linked, I misspoke when I mentioned in brackets (what was a passing thought) the variance component for "trt_grp" is non-interpretable (realized I had to edit that part out 1 minute later). You will not get "any" variance component for "trt_grp" at all to worry about its interpretation. You simply are using "trt_grp" (or "grp" in your case) to make your grouping variable (i.e., what appears to the right of `|`) more specific (e.g., combination of study-groups, etc.) for the levels of the variable appearing to the left of `|` (often taken to be the target of generalization). Thus, you can't generalize beyond the levels of "trt_grp" or the like unless you place "trt_grp" on the left of the `|`. To digress a bit, in the general multilevel modeling literature, it is debated what really is generalizable (the grouping variable, the varying intercepts/slopes, or the predictions/estimates of random effects). At any rate, we simply use an indexing variable (or what James referred to as an ID variable) as a grouping variable in whatever combinations to make our assumptions regarding true effects more specific. Remember that this also means that we are adding more parameters to the model. Thus, we may sometimes create but end up not using such ID variables if the data structure or size (or both) don't support encoding such specific assumptions about the random effects. Just my two cents, James will have more to say, for sure, Jack ps. I'm developing an R function to make it visually possible to see different study structures as well as their counts in meta-analytic datasets and hope that such an approach would make the modeling choices a tiny bit more clear (will share that in near future).
On Wed, Aug 4, 2021 at 12:42 PM Farzad Keyhan <f.keyhaniha at gmail.com> wrote:
For sure, much appreciated. I'll follow your comments in that thread, then. Fred On Wed, Aug 4, 2021 at 11:47 AM James Pustejovsky <jepusto at gmail.com> wrote:
I see. To keep things organized, I'll offer some comments in the thread that you linked above. On Wed, Aug 4, 2021 at 11:34 AM Farzad Keyhan <f.keyhaniha at gmail.com> wrote:
Hi James, Thank you for your informative response. I also appreciate Jack
Solomon's
helpful off-list comments (they may be helpful to other list members as well). To pick up on your last point which was that leaving the SCE's original random specification as is would imply that "true effect sizes at
different
time points are *fully* correlated if they come from the same study and
the
same category", one concern I have, as noted in my original email, is
that
most of my studies have multiple treatment groups in them. I realized that this issue has recently come up on the list (
My collaborators have only "for indexing purposes" distinguished between treatment groups in each study (variable "grp"). The post linked in the previous paragraph says: "it may be more "realistic" to assume that true effect sizes at different time points
are
correlated IF they come from the same treatment group in each study;
rather
than simply each study as a whole disregarding the groups." I agree (and perhaps you do too?) with that statement. In the case of my SCE model, my "time" random effect, then, has to change from: "~ time | interaction(study, mod_cat)" to now: "~ time | interaction(study, grp, mod_cat)". But as the follow-up answer (
also explains, this strategy will produce a perhaps non-interpretable variance component for "grp". So, I was wondering what your thoughts are on using this strategy (i.e., adding an indexing variable like "grp" in the random specification of "time")? All the best, Fred On Tue, Aug 3, 2021 at 10:24 PM James Pustejovsky <jepusto at gmail.com> wrote:
Hi Farzad, It's definitely possible to add further levels of random effects within each subgroup. (We didn't give examples of this in the paper due to
space
constraints.) I think the specification that you wrote in your original email could work with variety of different structures for the second set of random effects. For instance, struct = "CS" would imply a simple random
effect for
each study-category combination, struct = "HCS" would allow differing variance by time-point, or struct = "AR" would allow for an
auto-regressive
structure per time-point. The one drawback of these specifications is
that
the variance of the random effects is assumed to be constant across
moderator categories. That might or might not be reasonable--it's a
question of empirical fit.
Alternately, you could try something along the lines of
random = list(~ mod_cat | study, ~ mod_cat | interaction(study,time)),
struct = c("DIAG","DIAG")
which will give you a time-point specific random effect with a unique
variance component for every level of mod_cat.
Alternately, you could just leave it as the SCE model. You wrote: "we
want to encode the assumption that the true effect sizes at different
time
points are correlated if they come from the study and the same
category."
The regular SCE model (using random = ~ mod_cat | study, struct =
"DIAG")
already encodes that assumption. It implies that true effect sizes at different time points are *fully* correlated if they come from the same study and the same category, due to there being just one random effect
per
study-category. Kind Regards, James On Wed, Jul 21, 2021 at 3:42 PM Farzad Keyhan <f.keyhaniha at gmail.com> wrote:
Dear James (and the List Member), First of all, what a useful paper (congrats!). My colleagues and I
want
to fit separate SCE models to a number of categorical moderators in our longitudinal studies (data structure is below, note that multiple rows with the same time indicator are due to multiple treatment groups in each study, not any additional outcome measures). In your paper, although you talk about adding random effects per data particulars, you seem to mainly mean that for your CHE model. In our SCE models, we want to encode the assumption that the true
effect
sizes at different time points are correlated if they come from the
study
and the same category (*a bit unsure whether being from the same
category
is quite needed*).
(1) Does this assumption make sense given the purpose of the SCE model
that you had in mind?
(2) If yes, what choices of "struct=" would make sense (to be
empirically
confirmed)?
** mods = ~ mod_cat*time, random = list(~ mod_cat | study, ~ time |
interaction(study,mod_cat)), struct = c("DIAG","???") **
study mod_cat time
1 1 0
1 2 0
1 1 1
1 2 1
2 1 0
2 2 1
3 1 0
Thanks,
Fred
[[alternative HTML version deleted]]
_______________________________________________ R-sig-meta-analysis mailing list R-sig-meta-analysis at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
[[alternative HTML version deleted]]
_______________________________________________ R-sig-meta-analysis mailing list R-sig-meta-analysis at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis