Hi all,
Credibility intervals can be useful because ? in contrast to confidence
intervals ? they consider the heterogeneity of synthesized effects and make
predictions about the range of values within which the true effect of
future studies will likely fall (e.g., IntHout et al., 2016).
It is straightforward enough to calculate credibility intervals for rma and
rma.mv objects in metafor using the `predict` function. However, (a) how
would one compute credibility intervals for multivariate multilevel
meta-analytic models with cluster-robust variance estimation, and (b) is
doing so even meaningful, given that the covariance matrix has only been
aproximated? I included more details about both questions below.
Thanks a lot!
Fabian
-----
Regarding question (a), I would compute the credibility interval for a
simplified example as follows:
# fit model
vcv <- clubSandwich::impute_covariance_matrix(vi = data$vi, cluster =
data$cluster_id, r = 0.7)
m <- metafor::rma.mv(yi, vcv, random = ~ 1 | cluster_id/es_id, data =
data)
robu <- clubSandwich::coef_test(m, vcov = "CR2")
# compute 95% credibility interval
robu$beta - 1.96*sqrt(sum(m$sigma2) + robu$SE^2)
robu$beta + 1.96*sqrt(sum(m$sigma2) + robu$SE^2)
Regarding question (b), since V in this model is just a rough approximation
of the empirical covariance structure, the variance components of the model
are also just an approximation as well. Even when using cluster-robust
standard errors for the computation of the credibility intervals, we still
use the model's approximated variance components, and thus the credibility
interval provides just a rough guess of the range within which future
studies will fall.
[[alternative HTML version deleted]]