[R-meta] Multivariate meta-analysis with unknown covariances?

First of all: Thanks a lot to the two of you for your kind and very 
helpful answers!
What a fortunate coincidence that James Pustejovsky published his blog 
entry the same day I was asking how to deal with unknown covariances.
Does anyone know a published reference, in which the three steps are 
recommended? Until now, I couldn?t find one.

I want to share my R-Code with the list (some explanations included). 
Maybe someone more experienced might check if my specification is 
correct. And maybe it helps other clueless people with the same problem:

1. Calculation of the effect sizes:

data <- escalc(measure = "SMD", m1i = m12, sd1i = sd12, n1i = n12, m2i = 
m22, sd2i = sd22, n2i = n22, data = data, append = TRUE, replace = 
FALSE)
data <- escalc(measure = "PBIT", ai = a1, bi = b1, ci = c1, di = d1, 
data = data, append = TRUE, replace = FALSE)

I calculated the effect sizes with Hedges g (?SMD?) and transformed 
dichotomized data to standardized mean differences with the help of the 
probit transformed risk difference (?PBIT?).

2.  Imputation of the variance-covariance matrix:

Vlist <- impute_covariance_matrix(vi = data$vi, cluster = data$study, r 
= 0.7)

Right now, I fixed the correlation between all outcomes in the same 
study to 0.7. This is quite rough and I want to precise this guestimates 
(I asked most of the original autors if they can provide me with the 
correlations between the outcomes and will also precise this guestimate 
by substituting it with correlations from other studies that used the 
same outcomes).

3. Conduct the multivariate meta-analysis:

MultiMeta <- rma.mv(yi = yi, V = Vlist, mods =  ~ factor(controlgroup) 
-1, random = ~ factor(outcome)|study, struct = "CS", data = data)

I formulated a multivariate meta-analysis with random effects and 
included the imputed covariance matrix into the model. I had to fix the 
structure to a compound symmetric structure (?CS?), because with less 
restrictive structures I received the following warning message:
Fehler in rma.mv(yi = yi, V = VPostdicho, mods = ~factor(Kontrollgruppe) 
-  : Optimizer (nlminb) did not achieve convergence (convergence = 1).
Zus?tzlich: Warnmeldung: In .process.G.afterrmna(mf.g, g.nlevels, 
g.levels, struct[1], tau2, :Some combinations of the levels of the inner 
factor never occurred. Corresponding rho value(s) fixed to 0.
Probably the warning message is due to the fact that I have 24 different 
outcome measures for 28 effect sizes ? so there is no combination of 
outcomes that appears several times. In addition to that, I included a 
categorical moderator (mods =  ~ factor(controlgroup) -1) in the 
meta-analysis. When I set e.g. struct = ?UN?, the optimizer is not 
converging. With struct = ?CS?, it is.
So setting struct = ?CS? seems to be the only possiblity here?

4. Compute robust tests and confidence intervals:
I tried both options (in metafor as well as in club sandwich) to 
estimate robust standard errors and p-values:

metafor_robust <- robust.rma.mv(MultiMeta, cluster = data$study)
ClubSandwich_robust <- coef_test(MultiMeta, vcov = "CR2")

Both options result in similar (but not identical) values. Probably, for 
my research it is interchangeable which option I choose?

For further analysis (like Cook?s distance or Egger?s test) one uses 
exclusively the robust estimates, or?
In my case, the test for residual heterogenity in the model with the 
imputed covariance matrix is highly significant. When I exclude one 
effect size that is an outlier, the residual heterogenity is not 
significant anymore. May I present this result, although it refers to 
the model with the imputed covariance matrix? (There is no test for 
residual heterogenity for robust estimates)

Best regards,
Isabel Schlegel

Am 10.08.2017 21:23, schrieb Viechtbauer Wolfgang (SP):

[R-meta] Multivariate meta-analysis with unknown covariances?

Thread (6 messages)