Skip to content

[R-meta] adjusting individual studies for publication bias

3 messages · Filippo Gambarota, Wolfgang Viechtbauer

Original
Filippo Gambarota
# 
Hi!
I was wondering if there is a method, maybe using selection models, to
adjust the individual studies taking into account the publication bias. I
was thinking something like:

```
library(metafor)

k <- 500
tau2 <- 0.1
d <- 0.2
n1 <- n2 <- round(runif(k, 10, 300))
deltai <- rnorm(k, 0, sqrt(tau2))
yi <- rnorm(k, d, sqrt(1/n1 + 1/n2 + tau2))
vi <- (rchisq(k, n1 + n2 - 2) / (n1 + n2 - 2)) * (1/n1 + 1/n2)

sim <- data.frame(yi, vi)
sim <- escalc(yi = yi, vi = vi, data = sim)
sim <- summary(sim)
wi <- exp(-5 * sim$pval) # simulating selecting with negexp model
keep <- rbinom(nrow(sim), 1, wi) == 1 # keep if coin flip is 1
sims <- sim[keep, ]

res <- rma(yi, vi, data = sims)
sel <- selmodel(res, type = "negexp")

# linear combination of b + ranef

yi_adj <- sel$b[[1]] + rnorm(sel$k, 0, sqrt(sel$se^2 + sel$vi + sel$tau2))

plot(sims$yi, yi_adj)
```
But of course i'm not sure. Something like using b, tau2, vi and seb to
estimate the individual effect correcting for publication bias.
Thank you!
3 days later
Wolfgang Viechtbauer
# 
Hi Filippo,

First, a general note about your code: The p-values you get from summary(sim) are two-sided, but in selmodel(), you are using the (default) alternative="greater". If you use selmodel(res, type = "negexp", alternative = "two.sided"), then you will find that the delta=5 value can be recovered quite accurately.

As for your actual question: Your idea of using

sel$b[[1]] + rnorm(sel$k, 0, sqrt(sel$se^2 + sel$vi + sel$tau2))

is sensible, but this is more like a new sample from the same population (with identical sampling variances) than adjusting the actually observed estimates. But publication bias does not change the effects themselves - it changes *which* estimates we end up seeing in our sample. So I am not sure to what extent the idea of 'adjusting the observed effects for publication bias' makes sense in the first place.

Best,
Wolfgang
Filippo Gambarota
# 
Thank you Wolfgang for the correction about the p value. And yes, about the
idea of correcting the individual studies you are right. I was thinking
something related to the type-m error
https://library.virginia.edu/data/articles/assessing-type-s-and-type-m-errors
or something where a study contribute with a certain weight to an inflated
estimated effect size thus we could remove proportionally the inflated part
to obtain a set of studies where the weighted average is the pb corrected
parameter.
Thank you again!
Filippo

On Thu, May 8, 2025 at 3:06?PM Viechtbauer, Wolfgang (NP) <
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote: