[R-meta] rma.mv-When a higher level can't be modeled because of one row
For further clarity, each study uses only one level of the 10 available levels of "measure", except one study that uses two levels of the 10 available levels of "measure". The question is that if this one exceptional study was like all other studies, then, we could use: ~ 1 | measure / study ... . But now because of this one exceptional study, we can't use: ~ 1 | measure / study ... . So, to use measure as a random factor what should we do? One suggestion is to use: ~ 1 | study / measure ... . However, given that all studies, except one, agree with ~ 1 | measure / study..., is there a technical consequence to using ~ 1 | study / measure? Thank you for your help, Fred measure row study outcome 1 1 1 1 1 2 1 2 # 1 3 2 1 <--- measure on this row 2 4 2 1 1 5 3 1 1 6 3 2 2 7 4 1 9 8 5 2 9 9 6 4 ...
On Mon, Nov 8, 2021 at 11:53 AM Farzad Keyhan <f.keyhaniha at gmail.com> wrote:
Thanks, as I mentioned the following is just the structure. In the full dataset, "measure" has 10 unique levels. In 50 studies, only one "measure" is used for all rows of a study. But in one study two measures were used (this study has two rows, therefore, only one row of it has to change to allow one "measure" to take all its rows just like the other 50 studies). Does that make sense and does your advice still stand regarding either ignoring "measure" or using it as study/measure? measure row study outcome 1 1 1 1 1 2 1 2 # 1 3 2 1 <--- measure on this row 2 4 2 1 1 5 3 1 1 6 3 2 ... On Mon, Nov 8, 2021 at 11:38 AM Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
You can compute CIs with confint() and look at the profile likelihoods with profile(). Now I see that 'measure' has 10 different levels, not 2 (as I thought), so I am apparently not fully understanding the data structure. Best, Wolfgang
-----Original Message-----
From: Farzad Keyhan [mailto:f.keyhaniha at gmail.com]
Sent: Monday, 08 November, 2021 18:03
To: Viechtbauer, Wolfgang (SP)
Cc: R meta
Subject: Re: rma.mv-When a higher level can't be modeled because of one row
This is excellent advice, exactly what I was looking for. However, how
can an rma.mv() user find out if any of her/his variance estimates is
poor? Is there anything we can extract or compute from a fitted
object?
random = ~1 | measure/study/esID
estim sqrt nlvls fixed factor
sigma^2.1 0.0502 0.2242 10 no measure <-- this
must be very poor then
sigma^2.2 0.3016 0.5491 51 no measure/study
sigma^2.3 0.1547 0.3934 405 no measure/study/esID
On Mon, Nov 8, 2021 at 10:36 AM Viechtbauer, Wolfgang (SP)
<wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
On further though, I would then just ignore 'measure' altogether. Definitely do
not use it at the top of a random effects hierarchy as that would induce possible dependence for all 499 rows falling under measure = 1 based on a very poor variance estimate.
Best, Wolfgang
-----Original Message----- From: Farzad Keyhan [mailto:f.keyhaniha at gmail.com] Sent: Monday, 08 November, 2021 17:13 To: Viechtbauer, Wolfgang (SP) Cc: R meta Subject: Re: rma.mv-When a higher level can't be modeled because of one row The actual data has 500 rows. That is just the structure of the data that I showed. That is why I want to make sure if it is reasonable to ignore 499 rows that agree with "measure/study" and just base my random effect specification on one row that suggests "study/measure". In other words, I want to make sure my random-effects seem a bit more in line with the generality of my data rather than an exception that has occurred in just one row. Is there a consequence if I switch from "measure/study" to "study/measure" given this situation? On Mon, Nov 8, 2021 at 10:00 AM Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
Like I said before, if this is really all the data, then I wouldn't do any
of
that, because this will be a way too complex model for so little data.
-----Original Message----- From: Farzad Keyhan [mailto:f.keyhaniha at gmail.com] Sent: Monday, 08 November, 2021 16:51 To: Viechtbauer, Wolfgang (SP) Cc: R meta Subject: Re: rma.mv-When a higher level can't be modeled because of one row Sure, so, I shouldn't worry that all rows but one suggest "measure/study" and only because of that one exceptional row, do: "~ 1 | study/measure/outcome" or "~ 1 | study/outcome/measure"? Fred On Mon, Nov 8, 2021 at 9:39 AM Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
No, I meant using "~ 1 | study/measure/outcome" or "~ 1 |
study/outcome/measure".
Best, Wolfgang
-----Original Message----- From: Farzad Keyhan [mailto:f.keyhaniha at gmail.com] Sent: Monday, 08 November, 2021 16:22 To: Viechtbauer, Wolfgang (SP) Cc: R meta Subject: Re: rma.mv-When a higher level can't be modeled because of one
row
Thanks Wolfgang. Yes, this is just the data structure. Focusing on the "making [measure] nested within study" part of your suggestion, you mean in row # 3, I recode the "measure" value of 1 to 2, or even delete row # 3 altogether, or "~1 | measure/study/outcome" by default will take care of making "measure" nested in study? Thank you, Fred On Mon, Nov 8, 2021 at 4:13 AM Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
Dear Fred, I would consider using measure as a fixed effect or making it nested
within
study (or within outcome). But none of this might really be appropriate
for
a
dataset this small (but I assume this was just constructed for
illustrating
your
question).
Best, Wolfgang
-----Original Message----- From: Farzad Keyhan [mailto:f.keyhaniha at gmail.com] Sent: Friday, 05 November, 2021 2:59 To: R meta Cc: Viechtbauer, Wolfgang (SP) Subject: Re: rma.mv-When a higher level can't be modeled because of
one
row
For clarity, by a solution, I mean how can I account for the heterogeneity in true effects attributable to "measure", while "measure" is neither a perfect candidate for being the nestor of study: random = ~1 | measure/study/outcome nor a perfect candidate for being crossed with study: random = list(~1 | study/outcome, ~1|measure) Thank you, Fred On Thu, Nov 4, 2021 at 2:26 PM Farzad Keyhan <f.keyhaniha at gmail.com>
wrote:
Dear Experts, In my toy data below, if in row # 3, "measure" was 2 (instead of
1),
then, I could take "measure" as a level higher than study: random = ~1 | measure/study/outcome But right now, because in study 2 (rows # 3 and 4) "measure" can
vary,
"measure" can't be considered a level higher than study. On the other hand, because "measure" varies only in one study, I
can't
take "measure" as a crossed random-effect either. I was wondering what solutions the expert list members might have
for
this situation? Thanks, Fred measure row study outcome 1 1 1 1 1 2 1 2 # 1 3 2 1 <--- measure on this row 2 4 2 1 1 5 3 1 1 6 3 2