[R-meta] weight in rmv metafor

Of course the weights "impact the estimated fixed effects". But whether 
studies with multiple effect sizes tend to receive more weight depends on 
various factors, including the variances of the random effects and the 
sampling error (co)variances. 

A more detailed discussion around the way weighting works in rma.mv 
models can be found here: 

http://www.metafor-project.org/doku.php/tips:weights_in_rma.mv_models 

Note that weights(res, type="rowsum") currently only works in the 'devel' 
version of metafor, so follow 
https://wviechtb.github.io/metafor/#installation if you want to reproduce 
this part as well. 

I hope this clarifies things. 

Best, 
Wolfgang 

-----Original Message----- 
From: Huang Wu [mailto:huang.wu at wmich.edu] 
Sent: Sunday, 07 June, 2020 19:52 
To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis at r-project.org 
Subject: RE: [R-meta] weight in rmv metafor 

Dear Dr. Viechtbauer, 

Thank you very much for your helpful reply. 

To be clear, I wonder if the multivariate approach will downweight 

estimates 

from a study that contains multiple effect sizes? 
I saw in a previous posts (https://stat.ethz.ch/pipermail/r-help/2017- 
February/444703.html), your said, "if you fit an appropriate model to the 
data at hand, the 'default weights' used by rma.mv() will be just fine." 
Does that mean that weights in rma.mv model would not impact the 

estimated 

fixed effects? 
I found that in the forest plot I generate through forest(), studies with 
multiple effect sizes tend to have bigger weights. I also used weights() 

to 

check the weights given to each effect sizes and found the same thing (see 
below for my code). I wonder if the weights for each effect sizes 

presented 

in forest plot is correct? 

Thank you very much again for your help. 

Best wishes 
Huang 

Vt <- impute_covariance_matrix(vi = try$v, #known correlation vector 
cluster = try$ID, #study ID 
r = 0.80) #assumed correlation 

Mt <- rma.mv(yi=d, #effect size 
V = Vt, #variance (tHIS IS WHAt CHANGES FROM HEmodel) 
random = ~1 | ID/IID, #nesting structure 
test= "t", #use t-tests 
data=try, 
method="REML") 
weights(Mt) 

From: Viechtbauer, Wolfgang (SP) 
Sent: Sunday, June 7, 2020 6:56 AM 
To: Huang Wu; r-sig-meta-analysis at r-project.org 
Subject: RE: [R-meta] weight in rmv metafor 

Dear Huang, 

The weighting in rma.mv() models is more complex than in 'simple' models 
fitted with rma() (same as rma.uni()). Depending on the particular model 

you 

are fitting with rma.mv(), the model-implied marginal var-cov matrix of 

the 

estimates (which you can see with vcov(<model>, type="obs")) is not just a 
diagonal matrix (as is the case for rma() models), but also involves 
covariances. The inverse of this matrix is the weight matrix, which is 

then 

also not just a diagonal matrix. 

For example, when some studies contribute multiple estimates, we might 
consider fitting a multilevel/multivariate model with random effects for 
studies and random effects for estimates within studies. When the 

estimated 

between-study variance component is greater than zero, then this implies a 
certain amount of covariance for effects from the same study. This leads 

to 

negative off-diagonal elements in the weight matrix for estimates from the 
same study. As a result, if the ith study contributes k_i estimates, it is 
not treated as if there were k_i independent studies. 

This has been discussed in the past on this mailing list, so you might 

want 

to search the archives for some relevant posts. Googling for: 

site:https://stat.ethz.ch/pipermail/r-sig-meta-analysis/ rma.mv weights 

brings up some relevant posts. 

Roughly speaking, the robust variance estimation method works as follows. 

We 

start with a 'working model' that is hopefully some decent approximation 

to 

the true model and that also captures the dependencies in the estimates. 
This model provides us with the estimates of the fixed effects. However, 
because we might not be able to capture all dependencies correctly with 

this 

working model, the var-cov matrix of the estimated fixed effects might not 
be correct. Hence, based on the working model, we can use the robust 
variance estimation method to obtain a var-cov matrix that is 
(asymptotically) correct and use this for testing the fixed effects. 

Therefore, the robust variance estimation method does not actually lead to 
changes in the estimated fixed effects. Those are determined based on the 
working model. That is why coef_test() will give you the exact same 
estimates of the fixed effects as those from the working model you use as 
input to this function. 

That is why it is important to use a working model that is at least some 
decent approximation. While the fixed effects estimates might even be 
unbiased when using a really poor working model, the estimates will not be 
very efficient. 

Best, 
Wolfgang 

-----Original Message----- 
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r- 

project.org] 

On Behalf Of Huang Wu 
Sent: Sunday, 07 June, 2020 0:37 
To: r-sig-meta-analysis at r-project.org 
Subject: [R-meta] weight in rmv metafor 

Hi, all, 

I am conducting a multivariate meta-analysis using rmv in metaphor 

package. 

I wonder how rmv calculate weights for each effect sizes? I wonder if 
studies with more effect sizes get more total weights? 

I read an article saying "The robust variance estimation methods upweight 
effect sizes that are estimated with greater precision (due to 

differences 

in sample sizes, level of randomization, predictive power of covariates, 
etc.) and downweight estimates from studies that contribute multiple 

effect 

size estimates". (Kraft,Blazar, Hogan, 2018). Is that right? 

I am using rmv in metafor package to estimate the model and use coef_test 

in 

sandwich package to do significance test. Both give the same pooled 

effect 

sizes though. I understand that weights also impact pooled effect size 
estimate. In this case, how will robust variance estimation impact my 

weight 

mean effect size? Thanks 

Best wishes 
Huang