[R-meta] "impute-the-correlation + robustness + sensitive analysis" strategy
Celia, If you have reasonable prior information about the correlation between certain tests, which suggests that some tests are more highly correlated then others, then I would definitely recommend using that information. In principle, you can still use the "impute-the-correlation" strategy even while assuming unequal correlations between certain tests. For instance, as you suggested, you might assume r = 0.7 for the inter-correlation between T1 through T4, but then assume r = 0.2 for the correlation between these measures and T5. For sensitivity analysis, you could vary these correlations by adding/subtracting 0.1 or 0.2 from each (off-diagonal) cell. Another useful sensitivity analysis would be to assume zero correlations between tests from different domains and then also use the struct = "DIAG" argument in rma.mv. This amounts to estimating separate (marginal) models for each domain. If you get very different average effect estimates than you do with the full multivariate model, then it would indicate that the results are going to be fairly sensitive to the assumption you make about the cross-domain correlations. Of course, implementing these approaches takes a bit more work. One approach would be to create the variance-covariance matrices for each study "by hand," and then store them in a list that can be fed into rma.mv. This is tedious but might it be the easiest way to go. I have an idea for how to make the impute_covariance_matrix() function more helpful for your use-case, but it will be a week or two before I can get to it. James On Tue, Jan 23, 2018 at 6:16 AM, C?lia Sofia Moreira <
celiasofiamoreira at gmail.com> wrote:
Dear all, I am performing a meta-analysis for the first time, and I would be grateful if you could give me some recommendations / suggestions. I apologize for this long description: I have 20 different papers and, because some papers have more than one experiment, 23 experiments are considered, which correspond to 23 different samples. The complexity of my case is that each experiment includes different domains, and each domain includes several tests. These domains are latent variables in the individual studies (tests are the indicators). I'm interested in studying the effects in five different domains. Thus, for each domain, I usually collected more than one test type measure. Finally, each sample includes an experimental and a control group. The effects were estimated using Hedges' g (adjusted) and then calculating the difference y between groups. In these calculations, I followed this valuable example: http://www.metafor-project.org/doku.php/analyses:morris2008 Unfortunately, papers do not report correlations/covariances between test measures and, so, for each y, I only have the estimated variance v. So, my database looks like: ID ----- Paper --------- Sample ------- Domain ------- Test ------- y ---- v ---- etc. 01 ------- P1 ---------- S1 ----------- D1 ---------- T1 -------- ... 02 ------- P1 ---------- S1 ----------- D1 ---------- T2 -------- ... 03 ------- P1 ---------- S1 ----------- D2 ---------- T3 -------- ... 04 ------- P1 ---------- S1 ----------- D4 ---------- T4 -------- ... 05 ------- P1 ---------- S1 ----------- D4 ---------- T5 -------- ... 06 ------- P1 ---------- S1 ----------- D4 ---------- T6 -------- ... 07 ------- P2 ---------- S2 ----------- D2 ---------- T3 -------- ... 08 ------- P2 ---------- S2 ----------- D2 ---------- T7 -------- ... 09 ------- P2 ---------- S2 ----------- D2 ---------- T8 -------- ... 10 ------- P2 ---------- S2 ----------- D3 ---------- T9 -------- ... 11 ------- P2 ---------- S2 ----------- D4 ---------- T4 -------- ... 12 ------- P2 ---------- S2 ----------- D4 ---------- T5 -------- ... 13 ------- P2 ---------- S2 ----------- D5 ---------- T0 -------- ... 14 ------- P3 ---------- S3 ----------- D1 ---------- T1 -------- ... 15 ------- P3 ---------- S4 ----------- D1 ---------- T1 -------- ... 16 ------- P3 ---------- S4 ----------- D2 ---------- T3 -------- ... ..... I would like to assess the significance of these five domains' effects as well as their correlation, using a meta-analyst approach. D1 is the central domain. Thus, I thought about performing a multivariate multilevel random meta-analysis model and then using the correlation matrix between true effects provided in the output. Since I have no sampling covariances, I used the "impute-the-correlation" strategy, by James Pustejovsky, together with robust and sensitive analysis. Because I'm interested in the domains (latent variables), the R code I'm using is the following: Vlist <- impute_covariance_matrix(vi = dat$v, cluster = dat$Sample, r = 0.2) # Followed by r=.4, r=.6 and r=.8 meta1 <- rma.mv(y ~ 0 + Domain, V=Vlist, random = ~ Domain | Sample, struct = "UN", data = dat) meta1r <- robust(meta1, cluster = dat$Sample); summary(meta1r); r1<-cov2cor(meta1r$vb); R1 <- round(c1,2) Does it make sense to you? Do you recommend corrections? Introducing 2 moderators in the model, I got interesting results with this "impute-the-correlation"+robust+sensitive strategy (ranging r from .2 to .8), namely, the significance of D1 true effect. However, the "impute-the-correlation" assumes equal correlations across studies and across outcomes. I've been searching in literature, and I found some papers reporting a high correlation between D2, D3, D4, D5: r=-6-.7 between pairs, and r=.7-.8 intra-pairs. Their correlation with D1 is lower, and even lower for one of the pairs. Given this different range of correlations among the outcomes, do you think that the results I got with this strategy are truthful? Or do you think it is better to perform the analysis in different steps, for different subsets of variables in the multivariate multilevel model, for example, considering two at a time, or D1 with each pair separately? Thank you very much for your attention! [[alternative HTML version deleted]]
_______________________________________________ R-sig-meta-analysis mailing list R-sig-meta-analysis at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis