[R-meta] weight in rmv metafor

Dear Huang,

The weighting in rma.mv() models is more complex than in 'simple' models fitted with rma() (same as rma.uni()). Depending on the particular model you are fitting with rma.mv(), the model-implied marginal var-cov matrix of the estimates (which you can see with vcov(<model>, type="obs")) is not just a diagonal matrix (as is the case for rma() models), but also involves covariances. The inverse of this matrix is the weight matrix, which is then also not just a diagonal matrix.

For example, when some studies contribute multiple estimates, we might consider fitting a multilevel/multivariate model with random effects for studies and random effects for estimates within studies. When the estimated between-study variance component is greater than zero, then this implies a certain amount of covariance for effects from the same study. This leads to negative off-diagonal elements in the weight matrix for estimates from the same study. As a result, if the ith study contributes k_i estimates, it is not treated as if there were k_i independent studies.

This has been discussed in the past on this mailing list, so you might want to search the archives for some relevant posts. Googling for:

site:https://stat.ethz.ch/pipermail/r-sig-meta-analysis/ rma.mv weights

brings up some relevant posts.

Roughly speaking, the robust variance estimation method works as follows. We start with a 'working model' that is hopefully some decent approximation to the true model and that also captures the dependencies in the estimates. This model provides us with the estimates of the fixed effects. However, because we might not be able to capture all dependencies correctly with this working model, the var-cov matrix of the estimated fixed effects might not be correct. Hence, based on the working model, we can use the robust variance estimation method to obtain a var-cov matrix that is (asymptotically) correct and use this for testing the fixed effects.

Therefore, the robust variance estimation method does not actually lead to changes in the estimated fixed effects. Those are determined based on the working model. That is why coef_test() will give you the exact same estimates of the fixed effects as those from the working model you use as input to this function.

That is why it is important to use a working model that is at least some decent approximation. While the fixed effects estimates might even be unbiased when using a really poor working model, the estimates will not be very efficient.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org]
On Behalf Of Huang Wu
Sent: Sunday, 07 June, 2020 0:37
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] weight in rmv metafor

Hi, all,

I am conducting a multivariate meta-analysis using rmv in metaphor package.
I wonder how rmv calculate weights for each effect sizes? I wonder if
studies with more effect sizes get more total weights?

I read an article saying "The robust variance estimation methods upweight
effect sizes that are estimated with greater precision (due to differences
in sample sizes, level of randomization, predictive power of covariates,
etc.) and downweight estimates from studies that contribute multiple effect
size estimates". (Kraft,Blazar, Hogan, 2018). Is that right?

I am using rmv in metafor package to estimate the model and use coef_test in
sandwich package to do significance test. Both give the same pooled effect
sizes though. I understand that weights also impact pooled effect size
estimate. In this case, how will robust variance estimation impact my weight
mean effect size? Thanks

Best wishes
Huang