Dear friends, I want to calculate the I^2 for multilevel and multivariate model. I understand that Dr. Viechtbauer has provided an approach to calculate it. http://www.metafor-project.org/doku.php/tips:i2_multilevel_multivariate Taking multilevel model for example, the codes are below. dat <- dat.konstantopoulos2011 res <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat) res W <- diag(1/dat$vi) X <- model.matrix(res) P <- W - W %*% X %*% solve(t(X) %*% W %*% X) %*% t(X) %*% W 100 * sum(res$sigma2) / (sum(res$sigma2) + (res$k-res$p)/sum(diag(P))) 100 * res$sigma2 / (sum(res$sigma2) + (res$k-res$p)/sum(diag(P))) However, in another Youtube video (https://www.youtube.com/watch?v=rJjeRRf23L8&t=1134s). Dr. Polanin showed different code to calculate it (at 41:03 of the video; see below). I2_2 <- (multi>metafor$sigma2[1])/(multi.metafor$sigma[1]+multi.metafor$sigma[2]+mean(corr1$var)) I2_3 <- (multi>metafor$sigma2[2])/(multi.metafor$sigma[1]+multi.metafor$sigma[2]+mean(corr1$var)) The key difference I think is the way of calculating error variance. I tried two approaches but it seems they produced different results. I wonder which one is right or both of them are right? Thank you.
[R-meta] Calculating I-square for multilevel and multivariate models
2 messages · Huang Wu, Wolfgang Viechtbauer
Dear Huang Wu, Indeed, the two approaches differ in how one defines the 'typical' sampling variance. Neither is right or wrong. In fact, there are other suggestions out there, such as taking the harmonic mean of the sampling variances (instead of the arithmetic mean). I believe we had some discussions around this in the past on the mailing list, so you might want to search the archives for that. Best, Wolfgang
-----Original Message----- From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Huang Wu Sent: Thursday, 02 September, 2021 7:19 To: r-sig-meta-analysis at r-project.org Subject: [R-meta] Calculating I-square for multilevel and multivariate models Dear friends, I want to calculate the I^2 for multilevel and multivariate model. I understand that Dr. Viechtbauer has provided an approach to calculate it. http://www.metafor-project.org/doku.php/tips:i2_multilevel_multivariate Taking multilevel model for example, the codes are below. dat <- dat.konstantopoulos2011 res <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat) res W <- diag(1/dat$vi) X <- model.matrix(res) P <- W - W %*% X %*% solve(t(X) %*% W %*% X) %*% t(X) %*% W 100 * sum(res$sigma2) / (sum(res$sigma2) + (res$k-res$p)/sum(diag(P))) 100 * res$sigma2 / (sum(res$sigma2) + (res$k-res$p)/sum(diag(P))) However, in another Youtube video (https://www.youtube.com/watch?v=rJjeRRf23L8&t=1134s). Dr. Polanin showed different code to calculate it (at 41:03 of the video; see below). I2_2 <- (multi>metafor$sigma2[1])/(multi.metafor$sigma[1]+multi.metafor$sigma[2]+mean(corr 1$var)) I2_3 <- (multi>metafor$sigma2[2])/(multi.metafor$sigma[1]+multi.metafor$sigma[2]+mean(corr 1$var)) The key difference I think is the way of calculating error variance. I tried two approaches but it seems they produced different results. I wonder which one is right or both of them are right? Thank you.