Skip to content

[R-meta] Question on three-level meta-analysis

5 messages · David Pedrosa, Wolfgang Viechtbauer

David Pedrosa
# 
Dear list,

there is one question I have not been able to get my head around and 
it's about whether if estimation of variance-covariance-matrices in a 
nested/multlevel hierarchical model make sense. To put things in a 
context, we have ~60 studies for which we could estimate a standardised 
mean difference and these studies are with minor exceptions all 
independent. Yet, there are 6 categories of interventions with something 
between 2 and 30 studies nested within, so that we have individuals, 
studies and category_of_intervention. We also added two moderators in 
the model; quality of studies and whether it's a RCT or a NRCT which 
resulted in the following:

res <- rma.mv(yi, vi,
 ?? ???? ??? ??? random = ~ 1 | category/study_id,
 ?? ???? ??? ??? mods= ~ qualsyst*factor(study_type),
 ?? ???? ??? ??? data=dat)

If there were studies in which some participants received different 
treatments (i.e. many of them were not independent), I guess the 
estimation of a different vcov should make sense. But I think it's 
possibly only 3-5 studies within all 60 of them. So is it conceptually 
correct to estimate the vcov for the level 'category' and stick it into 
the model or is that already included as I use category as random 
effect? I don't think it makes a huge difference, but I'm not sure about it.

Thanks for your help,

David
Wolfgang Viechtbauer
# 
Dear David,

I don't quite understand your question. What variance-covariance-matrices are you referring to and how would you stick them into the model?

Best,
Wolfgang
David Pedrosa
# 
Sorry Wolfgang for not being clear. I was wondering if it makes sense to 
estimate variance-covariance-matrices for the level "category" as I was 
not sure whether this level is independent or not (altough most studies 
look at different subjects, the interventions within distinct 
"categories" may be very different and therefore distinct variance has 
to be assumed). My idea was to estimate vcov for the level "category" 
and include it in the model as input for V, although I am inclined to 
think that including catregory as random factor may account for this 
variability already. Is that correct?

Best,

David


Am 28.03.2022 um 12:08 schrieb Viechtbauer, Wolfgang (SP):
Wolfgang Viechtbauer
# 
I still don't quite understand how you want to compute a var-cov matrix for the level "category". One can compute covariances for estimates that are correlated. This isn't done at some 'level' but it is done for the estimates directly. The basic rule is this: The sampling errors of two estimates are dependent when there is overlap (either full or partial) in subjects that contribute information to their computation. See also this presentation:

https://www.wvbauer.com/lib/exe/fetch.php/talks:2021_viechtbauer_dortmund_workflow_ma.html

So, if there are two estimates for different 'categories' and they are based on the same subjects, then yes, you ideally try to compute or at least approximate the covariance between their sampling errors. Just adding a random effect at the 'category' level does not capture the dependency in the sampling errors, only the potential dependency in the underlying true effects.

Computing/approximating the covariance can be difficult, which is where cluster-robust inference methods come into play. Again, see the talk above for further details. Note that if you have the 'devel' version of metafor installed, at the very end, one can now just do:

robust(res, cluster=dat$study, clubSandwich=TRUE)

to get the cluster-robust results based on the clubSandwich methods.

Best,
Wolfgang
David Pedrosa
# 
your explanation was indeed very helpful. I misunderstood the concept of 
covariance (matrices) and I was assuming that it applies to a higher 
level within the model, namely when studies are from the same category. 
Whereas your link and explanation helped me to understand where I was 
wrong.

I will now use the covariance estimation to account for those studies in 
which patients received more than one treatment, i.e. where sampling 
errors are dependent.

robust(res, cluster=dat$study, clubSandwich=TRUE)

Is a good place to start.

Best,

David


Am 29.03.2022 um 12:45 schrieb Viechtbauer, Wolfgang (SP):