[R-meta] Unrealistic confidence limits for heterogeneity?

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On
Behalf Of James Pustejovsky via R-sig-meta-analysis
Sent: Thursday, 30 March, 2023 19:00
To: R Special Interest Group for Meta-Analysis
Cc: James Pustejovsky
Subject: Re: [R-meta] Unrealistic confidence limits for heterogeneity?

Some further comments in addition to Michael's response below.

James

1. It is possible to allow for negative heterogeneity estimates using the
metafor package. Here is an example of the syntax:

library(metafor)

# generate data with no heterogeneity
set.seed(20230330)
k <- 10
vi <- 4 / (rpois(k, 22) + 8)
yi <- rnorm(k, mean = 0.2, sd = sqrt(vi))
dat <- data.frame(yi, vi)

# regular random effects meta-analysis, REML estimator
res1 <- rma(yi = yi, vi = vi, data=dat)
res1

# allow negative heterogeneity, REML estimator
res2 <- rma(yi = yi, vi = vi, data=dat, control=list(tau2.min=-min(vi)))
res2

# allow negative heterogeneity, other heterogeneity estimators
rma(yi = yi, vi = vi, data=dat, method = "ML",
control=list(tau2.min=-min(vi)))
rma(yi = yi, vi = vi, data=dat, method = "DL",
control=list(tau2.min=-min(vi)))
rma(yi = yi, vi = vi, data=dat, method = "HE",
control=list(tau2.min=-min(vi)))

2. You can obtain the estimated standard error for tau-squared as follows:
res1$se.tau2
res2$se.tau2

3. The metafor package implements several different confidence intervals
for tau-squared. The GENQ method requires estimating the model with method
GENQ.
confint(res1) # confidence interval for tau-squared
confint(res1, type = "PL") # profile likelihood method
confint(res1, type = "QP") # Q-profile method
rma(yi = yi, vi = vi, data=dat, weights = 1 / vi, method = "GENQ")  |>
 confint(type = "GENQ") # Generalized Q-statistic method