I still don't quite understand how you want to compute a var-cov matrix for the level "category". One can compute covariances for estimates that are correlated. This isn't done at some 'level' but it is done for the estimates directly. The basic rule is this: The sampling errors of two estimates are dependent when there is overlap (either full or partial) in subjects that contribute information to their computation. See also this presentation:
https://www.wvbauer.com/lib/exe/fetch.php/talks:2021_viechtbauer_dortmund_workflow_ma.html
So, if there are two estimates for different 'categories' and they are based on the same subjects, then yes, you ideally try to compute or at least approximate the covariance between their sampling errors. Just adding a random effect at the 'category' level does not capture the dependency in the sampling errors, only the potential dependency in the underlying true effects.
Computing/approximating the covariance can be difficult, which is where cluster-robust inference methods come into play. Again, see the talk above for further details. Note that if you have the 'devel' version of metafor installed, at the very end, one can now just do:
robust(res, cluster=dat$study, clubSandwich=TRUE)
to get the cluster-robust results based on the clubSandwich methods.
Best,
Wolfgang
-----Original Message-----
From: David Pedrosa [mailto:pedrosac at staff.uni-marburg.de]
Sent: Monday, 28 March, 2022 13:11
To: Viechtbauer, Wolfgang (SP);r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Question on three-level meta-analysis
Sorry Wolfgang for not being clear. I was wondering if it makes sense to estimate
variance-covariance-matrices for the level "category" as I was not sure whether
this level is independent or not (altough most studies look at different
subjects, the interventions within distinct "categories" may be very different
and therefore distinct variance has to be assumed). My idea was to estimate vcov
for the level "category" and include it in the model as input for V, although I
am inclined to think that including catregory as random factor may account for
this variability already. Is that correct?
Best,
David
Am 28.03.2022 um 12:08 schrieb Viechtbauer, Wolfgang (SP):
Dear David,
I don't quite understand your question. What variance-covariance-matrices are you
referring to and how would you stick them into the model?
Best,
Wolfgang
-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On
Behalf Of David Pedrosa
Sent: Monday, 28 March, 2022 10:02
To:r-sig-meta-analysis at r-project.org
Subject: [R-meta] Question on three-level meta-analysis
Dear list,
there is one question I have not been able to get my head around and
it's about whether if estimation of variance-covariance-matrices in a
nested/multlevel hierarchical model make sense. To put things in a
context, we have ~60 studies for which we could estimate a standardised
mean difference and these studies are with minor exceptions all
independent. Yet, there are 6 categories of interventions with something
between 2 and 30 studies nested within, so that we have individuals,
studies and category_of_intervention. We also added two moderators in
the model; quality of studies and whether it's a RCT or a NRCT which
resulted in the following:
res <- rma.mv(yi, vi,
?? ???? ??? ??? random = ~ 1 | category/study_id,
?? ???? ??? ??? mods= ~ qualsyst*factor(study_type),
?? ???? ??? ??? data=dat)
If there were studies in which some participants received different
treatments (i.e. many of them were not independent), I guess the
estimation of a different vcov should make sense. But I think it's
possibly only 3-5 studies within all 60 of them. So is it conceptually
correct to estimate the vcov for the level 'category' and stick it into
the model or is that already included as I use category as random
effect? I don't think it makes a huge difference, but I'm not sure about it.
Thanks for your help,
David