[R-meta] rma.mv-When a higher level can't be modeled because of one row

You can compute CIs with confint() and look at the profile likelihoods with profile().

Now I see that 'measure' has 10 different levels, not 2 (as I thought), so I am apparently not fully understanding the data structure.

Best,
Wolfgang

-----Original Message-----
From: Farzad Keyhan [mailto:f.keyhaniha at gmail.com]
Sent: Monday, 08 November, 2021 18:03
To: Viechtbauer, Wolfgang (SP)
Cc: R meta
Subject: Re: rma.mv-When a higher level can't be modeled because of one row

This is excellent advice, exactly what I was looking for. However, how
can an rma.mv() user find out if any of her/his variance estimates is
poor? Is there anything we can extract or compute from a fitted
object?

random = ~1 | measure/study/esID

           estim    sqrt  nlvls  fixed              factor
sigma^2.1  0.0502  0.2242     10     no             measure  <-- this
must be very poor then
sigma^2.2  0.3016  0.5491     51     no       measure/study
sigma^2.3  0.1547  0.3934    405     no  measure/study/esID

On Mon, Nov 8, 2021 at 10:36 AM Viechtbauer, Wolfgang (SP)
<wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

On further though, I would then just ignore 'measure' altogether. Definitely do

not use it at the top of a random effects hierarchy as that would induce possible
dependence for all 499 rows falling under measure = 1 based on a very poor
variance estimate.

Best,
Wolfgang

-----Original Message-----
From: Farzad Keyhan [mailto:f.keyhaniha at gmail.com]
Sent: Monday, 08 November, 2021 17:13
To: Viechtbauer, Wolfgang (SP)
Cc: R meta
Subject: Re: rma.mv-When a higher level can't be modeled because of one row

The actual data has 500 rows. That is just the structure of the data
that I showed. That is why I want to make sure if it is reasonable to
ignore 499 rows that agree with "measure/study" and just base my
random effect specification on one row that suggests "study/measure".

In other words, I want to make sure my random-effects seem a bit more
in line with the generality of my data rather than an exception that
has occurred in just one row.

Is there a consequence if I switch from "measure/study" to
"study/measure" given this situation?

On Mon, Nov 8, 2021 at 10:00 AM Viechtbauer, Wolfgang (SP)
<wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

Like I said before, if this is really all the data, then I wouldn't do any

of

that, because this will be a way too complex model for so little data.

-----Original Message-----
From: Farzad Keyhan [mailto:f.keyhaniha at gmail.com]
Sent: Monday, 08 November, 2021 16:51
To: Viechtbauer, Wolfgang (SP)
Cc: R meta
Subject: Re: rma.mv-When a higher level can't be modeled because of one row

Sure, so, I shouldn't worry that all rows but one suggest
"measure/study" and only because of that one exceptional row, do: "~ 1
| study/measure/outcome" or "~ 1 | study/outcome/measure"?

Fred

On Mon, Nov 8, 2021 at 9:39 AM Viechtbauer, Wolfgang (SP)
<wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

No, I meant using "~ 1 | study/measure/outcome" or "~ 1 |

study/outcome/measure".

Best,
Wolfgang

-----Original Message-----
From: Farzad Keyhan [mailto:f.keyhaniha at gmail.com]
Sent: Monday, 08 November, 2021 16:22
To: Viechtbauer, Wolfgang (SP)
Cc: R meta
Subject: Re: rma.mv-When a higher level can't be modeled because of one

row

Thanks Wolfgang.

Yes, this is just the data structure. Focusing on the "making
[measure] nested within study" part of your suggestion, you mean in
row # 3, I recode the "measure" value of 1 to 2, or even delete row #
3 altogether, or "~1 | measure/study/outcome" by default will take
care of making "measure" nested in study?

Thank you,
Fred

On Mon, Nov 8, 2021 at 4:13 AM Viechtbauer, Wolfgang (SP)
<wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

Dear Fred,

I would consider using measure as a fixed effect or making it nested

within

study (or within outcome). But none of this might really be appropriate

for

a

dataset this small (but I assume this was just constructed for

illustrating

your

question).

Best,
Wolfgang

-----Original Message-----
From: Farzad Keyhan [mailto:f.keyhaniha at gmail.com]
Sent: Friday, 05 November, 2021 2:59
To: R meta
Cc: Viechtbauer, Wolfgang (SP)
Subject: Re: rma.mv-When a higher level can't be modeled because of

one

row

For clarity, by a solution, I mean how can I account for the
heterogeneity in true effects attributable to "measure", while
"measure" is neither a perfect candidate for being the nestor of
study:

random = ~1 | measure/study/outcome

nor a perfect candidate for being crossed with study:

random = list(~1 | study/outcome, ~1|measure)

Thank you,
Fred

On Thu, Nov 4, 2021 at 2:26 PM Farzad Keyhan <f.keyhaniha at gmail.com>

wrote:

Dear Experts,

In my toy data below, if in row # 3, "measure" was 2 (instead of

1),

then, I could take "measure" as a level higher than study:

random = ~1 | measure/study/outcome

But right now, because in study 2 (rows # 3 and 4) "measure" can

vary,

"measure" can't be considered a level higher than study.

On the other hand, because "measure" varies only in one study, I

can't

take "measure" as a crossed random-effect either.

I was wondering what solutions the expert list members might have

for

this situation?

Thanks,
Fred

   measure       row   study   outcome
   1             1     1       1
   1             2     1       2
#  1             3     2       1 <--- measure on this row
   2             4     2       1
   1             5     3       1
   1             6     3       2