[R-meta] Clarification on ranef.rma.mv()

Please see below for my comments.

-----Original Message-----
From: Luke Martinez [mailto:martinezlukerm at gmail.com]
Sent: Tuesday, 14 September, 2021 19:25
To: Viechtbauer, Wolfgang (SP)
Cc: R meta
Subject: Re: [R-meta] Clarification on ranef.rma.mv()

Hi Wolfgang,

Thank you. And since in rma.mv() we can have up to two ~ inner | outer
random terms, then, I'm assuming to get the proportion of variation
for the second ~ inner | outer random term, I can do:

sds <- svd(chol(rma.mv_model4$H))$d
sds^2 / sum(sds^2)

Correct.

I guess one potential problem I'm running into is that what should we
do if we see that the proportion of explained between-studies variance
by only one or two levels of a categorical variable is almost zero
while rest of the levels of that categorical variable make significant
contributions?

The reason I ask this is that with continuous variables (using struct
= "GEN"), if a variable's contribution is almost zero, then, you can
decide not to use that continuous variable in the random part at all
(that variable altogether is overfitted).

But with categorical variables, when several levels make good
contributions to the between-studies variance except just one or two
levels, then, you can't easily decide not to use that whole
categorical variable in the random part at all.

Do you have any opinion on this dilemma?

I would choose a random effects structure that is motivated by the structure of the data and the possible sources of heterogeneity/variability/dependencies that I think may exist in the data. For example, if a slope may vary across units, then I would add a random effect for that slope to the model. If it turns out that the estimate of the slope variability is very low, dropping that random effect (which is the same as assuming that the variance is 0) or not will pretty much do the same thing. My preference would be not to change the model, because I generally try to avoid making changes to a model (since the consequences of such data dependent modifications are hard to predict).

The same applies to structures like "UN". If a particular tau^2 value is low, then no, I would not drop that random effect. You *can* however set particular tau^2 values to 0 (the 'tau2' argument of rma.mv() allows you to do that), but again, I would personally avoid doing that.

Best,
Wolfgang