Celia,
If you have reasonable prior information about the correlation between
certain tests, which suggests that some tests are more highly correlated
then others, then I would definitely recommend using that information.
In principle, you can still use the "impute-the-correlation" strategy even
while assuming unequal correlations between certain tests. For instance, as
you suggested, you might assume r = 0.7 for the inter-correlation between
T1 through T4, but then assume r = 0.2 for the correlation between these
measures and T5. For sensitivity analysis, you could vary these
correlations by adding/subtracting 0.1 or 0.2 from each (off-diagonal)
cell.
Another useful sensitivity analysis would be to assume zero correlations
between tests from different domains and then also use the struct = "DIAG"
argument in rma.mv. This amounts to estimating separate (marginal) models
for each domain. If you get very different average effect estimates than
you do with the full multivariate model, then it would indicate that the
results are going to be fairly sensitive to the assumption you make about
the cross-domain correlations.
Of course, implementing these approaches takes a bit more work. One
approach would be to create the variance-covariance matrices for each study
"by hand," and then store them in a list that can be fed into rma.mv.
This is tedious but might it be the easiest way to go. I have an idea for
how to make the impute_covariance_matrix() function more helpful for your
use-case, but it will be a week or two before I can get to it.
James
On Tue, Jan 23, 2018 at 6:16 AM, C?lia Sofia Moreira <
celiasofiamoreira at gmail.com> wrote:
Dear all,
I am performing a meta-analysis for the first time, and I would be
grateful
if you could give me some recommendations / suggestions. I apologize for
this long description:
I have 20 different papers and, because some papers have more than one
experiment, 23 experiments are considered, which correspond to 23
different
samples. The complexity of my case is that each experiment includes
different domains, and each domain includes several tests. These domains
are latent variables in the individual studies (tests are the indicators).
I'm interested in studying the effects in five different domains. Thus,
for
each domain, I usually collected more than one test type measure.
Finally, each sample includes an experimental and a control group. The
effects were estimated using Hedges' g (adjusted) and then calculating the
difference y between groups. In these calculations, I followed this
valuable example:
http://www.metafor-project.org/doku.php/analyses:morris2008
Unfortunately, papers do not report correlations/covariances between test
measures and, so, for each y, I only have the estimated variance v.
So, my database looks like:
ID ----- Paper --------- Sample ------- Domain ------- Test ------- y ----
v ---- etc.
01 ------- P1 ---------- S1 ----------- D1 ---------- T1 -------- ...
02 ------- P1 ---------- S1 ----------- D1 ---------- T2 -------- ...
03 ------- P1 ---------- S1 ----------- D2 ---------- T3 -------- ...
04 ------- P1 ---------- S1 ----------- D4 ---------- T4 -------- ...
05 ------- P1 ---------- S1 ----------- D4 ---------- T5 -------- ...
06 ------- P1 ---------- S1 ----------- D4 ---------- T6 -------- ...
07 ------- P2 ---------- S2 ----------- D2 ---------- T3 -------- ...
08 ------- P2 ---------- S2 ----------- D2 ---------- T7 -------- ...
09 ------- P2 ---------- S2 ----------- D2 ---------- T8 -------- ...
10 ------- P2 ---------- S2 ----------- D3 ---------- T9 -------- ...
11 ------- P2 ---------- S2 ----------- D4 ---------- T4 -------- ...
12 ------- P2 ---------- S2 ----------- D4 ---------- T5 -------- ...
13 ------- P2 ---------- S2 ----------- D5 ---------- T0 -------- ...
14 ------- P3 ---------- S3 ----------- D1 ---------- T1 -------- ...
15 ------- P3 ---------- S4 ----------- D1 ---------- T1 -------- ...
16 ------- P3 ---------- S4 ----------- D2 ---------- T3 -------- ...
.....
I would like to assess the significance of these five domains' effects as
well as their correlation, using a meta-analyst approach. D1 is the
central
domain. Thus, I thought about performing a multivariate multilevel random
meta-analysis model and then using the correlation matrix between true
effects provided in the output. Since I have no sampling covariances, I
used the "impute-the-correlation" strategy, by James Pustejovsky, together
with robust and sensitive analysis. Because I'm interested in the domains
(latent variables), the R code I'm using is the following:
Vlist <- impute_covariance_matrix(vi = dat$v, cluster = dat$Sample, r =
0.2) # Followed by r=.4, r=.6 and r=.8
meta1 <- rma.mv(y ~ 0 + Domain, V=Vlist, random = ~ Domain | Sample,
struct
= "UN", data = dat)
meta1r <- robust(meta1, cluster = dat$Sample); summary(meta1r);
r1<-cov2cor(meta1r$vb); R1 <- round(c1,2)
Does it make sense to you? Do you recommend corrections?
Introducing 2 moderators in the model, I got interesting results with this
"impute-the-correlation"+robust+sensitive strategy (ranging r from .2 to
.8), namely, the significance of D1 true effect.
However, the "impute-the-correlation" assumes equal correlations across
studies and across outcomes. I've been searching in literature, and I
found
some papers reporting a high correlation between D2, D3, D4, D5: r=-6-.7
between pairs, and r=.7-.8 intra-pairs. Their correlation with D1 is
lower,
and even lower for one of the pairs.
Given this different range of correlations among the outcomes, do you
think
that the results I got with this strategy are truthful? Or do you think it
is better to perform the analysis in different steps, for different
subsets
of variables in the multivariate multilevel model, for example,
considering
two at a time, or D1 with each pair separately?
Thank you very much for your attention!
[[alternative HTML version deleted]]