[R-meta] possible miscalculation of Cook’s distances

Dear Antonina,

An effect estimate whose standardized deleted residual falls beyond
+/-1.96 doesn't necessarily need to have a cook's distance
(influence), and/or a hat value (if additional moderators are used)
that are likewise extreme.

As a general proposition, it may be methodologically more reasonable
to simultaneously consult all these indices to flag an extreme effect
estimate.

That said, in your case, it does seem that the estimate from study 1,
experiment 1 has a large standardized deleted residual as well as a
large Cook's distance relative to that of other effect estimates. So,
I don't think there is any bug in any of the metafor functions you
used.

Below is a bit of code to catch that outlying effect estimate.

Kind regards,
Reza

dat <- data.frame(study=c(1,1,2,3,3,3), experiment=c(1:6),
                  yi=c( 68, 18, 31,20,10,26),
                  vi=c(60,32, 15, 19, 41, 82))

res.ml <- rma.mv(yi, vi,
                 random = ~ 1 | study/experiment,
                 data=dat,
                 slab = paste("Study ", study,", ", "Experiment ",
                              experiment, sep = ""))

dat <- transform(dat, obs = res.ml$slab)

outlier_limit <- qnorm(.975)

cook <- cooks.distance.rma.mv(res.ml,
                              reestimate = FALSE)

st_del_res_z <- rstudent.rma.mv(res.ml,
                                reestimate = FALSE)$z

cook_limit <- max(mean(range(cook)), boxplot.stats(cook, coef = 1.5)$stats[5])

i <- abs(st_del_res_z) > outlier_limit
k <- cook > cook_limit
oo <- which(i & k)

LL <- names(oo)

removed <- subset(dat, obs %in% LL)
new_dat <- subset(dat, !obs %in% LL)

On Thu, Feb 3, 2022 at 3:36 PM Antonina Dolgorukova

[R-meta] possible miscalculation of Cook’s distances

Thread (8 messages)