[R-meta] Error in weight effect in rma

Tue, Dec 7, 2021 6:30 AM

Hi Maria,

The point sizes are based only on the diagonal of the weight matrix. However, in such models, the weighting is more complex as described here:

https://www.metafor-project.org/doku.php/tips:weights_in_rma.mv_models

which covers this specific model (for other types of rma.mv() models, the weighting structure/matrix is of course different). So you might want to examine the 'row-sum weights' and you could even use them to specify the point sizes (via the psize argument) if you like.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On
Behalf Of Wiertsema, M.
Sent: Tuesday, 07 December, 2021 14:21
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] Error in weight effect in rma

Hi everyone,

I am currently working on a meta-analysis on the longitudinal
association between bullying (predictor) and social status (outcome).

Because we have multiple effects within one study, we used the
random-effects model for the forest plots. I have plotted the forest
plot, with effects based on Fisher's z, with the following code:
# random-effects model
ram2 <- rma.mv(yi, vi, random = ~ 1 | Sampleshort/Effectsize,
data=dat2, slab=paste(Effectsize, AuthorsR, YEAR, Country, plotn,
TypebullyingR, AgeBullyR, sep="; "), method="REML")
print(ram2, digits=3)

#check profile plots

par(mfrow=c(2,1))
profile(ram2, sigma2=1)
profile(ram2, sigma2=2)

#forest plots
forest(ram2, xlim=c(-0.85, 1.20), at=(c(-0.3, -0.2, -0.1, 0, 0.1, 0.2,
0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1)),
      rows=c(1:31),
      xlab="Fisher's Z", mlab="")

### add text with Q-value, dfs, and p-value
text(-0.85, -1, pos=4, cex=1, bquote(paste("Multilevel RE Model for
All Studies (Q = ",
                                          .(formatC(ram2$QE,
digits=2, format="f")), ", df = ",
                                          .(ram2$k - ram2$p),
                                          ", p = ",
.(formatC(ram2$QEp, digits=2, format="f")),
                                          ")")))

### switch to bold font
par(font=2)

### add column headings to the plot
text((-0.43), 32.5, "Effect number, Study, Country, Sample size, Type
of bullying, & Age bully")
text((1.015), 32.5, "Fisher's z correlations [95% CI]")

So far, so good.

However, when interpreting the forest plot we noticed something strange:
There are a couple of effect sizes, from two different samples, which
we expected to have a greater weight based on their CI and based on
the fact that those were the only effect sizes based on a sample. I
included the forest plot as an appendix, I hope you can see it.

For effect number 49 and 56, it would make sense to have a greater
weight based on their CI's, [0.45-0.57] and [0.36-0.44] respectively,
thus relatively small. For effect number 50 and 51, which are both
based on the same sample, the CI's are [0.24-0.36] and [0.34-0.46], so
the same interval as effect number 49 and even larger than effect
number 56. However, those two effects have a bigger weight compared to
effect number 49 and 46.

Additionally, the Cooks distance plot does recognize effect number 49
as an outlier, but based on the weight of that effect size in the
forest plot you would not expect that.

Am I missing something or is something weird going on in my code?

Please let me know if anything is unclear.
Many thanks in advance!

Best,
Maria

--
PhD student
University of Groningen, Faculty of Behavioural and Social Sciences
Grote Rozenstraat 38, 9712 TJ Groningen
email: m.wiertsema at rug.nl | phone: 050 - 363 3182

[R-meta] Error in weight effect in rma

Thread (2 messages)