Dear all, I have a question about retrieving or calculating the H2 statistic in the context of fitting a multilevel (random effects) model with rma.mv. Can the H2 statistic be calculated or retrieved based on rma.mv output? I cannot seem to find it in the rma.mv output. My apologies if I am just plainly overseeing it. Or does it actually not make sense to calculate it in the context of a multilevel model, considering it is not straightforward which heterogeneity variance to use? To provide a reproducible example: library(metafor) dat <- dat.konstantopoulos2011 res2 <- rma.mv(yi, vi, random = ~ 1 | district/study, data=dat) vs <- ((res2$k-1)* sum(1/res2$vi, na.rm = TRUE))/((sum(1/res2$vi, na.rm = TRUE))^2-sum((1/res2$vi)^2, na.rm = TRUE)) H2 <- (res2$sigma2[1] + vs)/vs Does the above make sense (I followed the equation on p. 250 of the metafor reference manual)? Or is the H2 easily retrievable in another way? Thank you and all the best, Marlene ______________________________________________________ VUmc disclaimer : www.vumc.nl/disclaimer AMC disclaimer : www.amc.nl/disclaimer
[R-meta] H2 in the context of rma.mv
3 messages · Wolfgang Viechtbauer, Werner, M.A. (Marlene)
Dear Marlene, It's not there, but you can calculate it manually. You can find a discussion on computing I^2 for more complex models here: https://www.metafor-project.org/doku.php/tips:i2_multilevel_multivariate Using the same principles discussed there, one can also compute H^2. And yes, your calculation below is doing just that. Using the notation in the page above, H^2 for the Konstantopoulos example would be: W <- diag(1/res$vi) X <- model.matrix(res) P <- W - W %*% X %*% solve(t(X) %*% W %*% X) %*% t(X) %*% W (sum(res$sigma2) + (res$k-res$p)/sum(diag(P))) / ((res$k-res$p)/sum(diag(P))) Note that I put sum(res$sigma2) in the numerator instead of just res$sigma2[1]. I think for H^2, that is more sensible, since we are asking how much larger the *total variance* is compared to the amount of variance expected based on sampling variability alone. Just putting part of the variance components in the numerator leads to a rather convoluted interpretation of what the H^2 value represents. Best, Wolfgang
-----Original Message----- From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Werner, M.A. (Marlene) Sent: Thursday, 17 March, 2022 18:49 To: r-sig-meta-analysis at r-project.org Subject: [R-meta] H2 in the context of rma.mv Dear all, I have a question about retrieving or calculating the H2 statistic in the context of fitting a multilevel (random effects) model with rma.mv. Can the H2 statistic be calculated or retrieved based on rma.mv output? I cannot seem to find it in the rma.mv output. My apologies if I am just plainly overseeing it. Or does it actually not make sense to calculate it in the context of a multilevel model, considering it is not straightforward which heterogeneity variance to use? To provide a reproducible example: library(metafor) dat <- dat.konstantopoulos2011 res2 <- rma.mv(yi, vi, random = ~ 1 | district/study, data=dat) vs <- ((res2$k-1)* sum(1/res2$vi, na.rm = TRUE))/((sum(1/res2$vi, na.rm = TRUE))^2-sum((1/res2$vi)^2, na.rm = TRUE)) H2 <- (res2$sigma2[1] + vs)/vs Does the above make sense (I followed the equation on p. 250 of the metafor reference manual)? Or is the H2 easily retrievable in another way? Thank you and all the best, Marlene
Thank you Wolfgang! The reference and rewritten calculation are very helpful. All the best, Marlene -----Oorspronkelijk bericht----- Van: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl> Verzonden: vrijdag 18 maart 2022 10:58 Aan: Werner, M.A. (Marlene) <m.a.werner at amsterdamumc.nl>; r-sig-meta-analysis at r-project.org Onderwerp: RE: H2 in the context of rma.mv Dear Marlene, It's not there, but you can calculate it manually. You can find a discussion on computing I^2 for more complex models here: https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.metafor-project.org%2Fdoku.php%2Ftips%3Ai2_multilevel_multivariate&data=04%7C01%7Cm.a.werner%40amsterdamumc.nl%7Ce44e912f5cbd4865757008da08c5e292%7C68dfab1a11bb4cc6beb528d756984fb6%7C0%7C0%7C637831943240660937%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=eXYGwfpD5lzaehVitYQjgFMIi7nAGEOyvvYUhOPnkMg%3D&reserved=0 Using the same principles discussed there, one can also compute H^2. And yes, your calculation below is doing just that. Using the notation in the page above, H^2 for the Konstantopoulos example would be: W <- diag(1/res$vi) X <- model.matrix(res) P <- W - W %*% X %*% solve(t(X) %*% W %*% X) %*% t(X) %*% W (sum(res$sigma2) + (res$k-res$p)/sum(diag(P))) / ((res$k-res$p)/sum(diag(P))) Note that I put sum(res$sigma2) in the numerator instead of just res$sigma2[1]. I think for H^2, that is more sensible, since we are asking how much larger the *total variance* is compared to the amount of variance expected based on sampling variability alone. Just putting part of the variance components in the numerator leads to a rather convoluted interpretation of what the H^2 value represents. Best, Wolfgang
-----Original Message----- From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Werner, M.A. (Marlene) Sent: Thursday, 17 March, 2022 18:49 To: r-sig-meta-analysis at r-project.org Subject: [R-meta] H2 in the context of rma.mv Dear all, I have a question about retrieving or calculating the H2 statistic in the context of fitting a multilevel (random effects) model with rma.mv. Can the H2 statistic be calculated or retrieved based on rma.mv output? I cannot seem to find it in the rma.mv output. My apologies if I am just plainly overseeing it. Or does it actually not make sense to calculate it in the context of a multilevel model, considering it is not straightforward which heterogeneity variance to use? To provide a reproducible example: library(metafor) dat <- dat.konstantopoulos2011 res2 <- rma.mv(yi, vi, random = ~ 1 | district/study, data=dat) vs <- ((res2$k-1)* sum(1/res2$vi, na.rm = TRUE))/((sum(1/res2$vi, na.rm = TRUE))^2-sum((1/res2$vi)^2, na.rm = TRUE)) H2 <- (res2$sigma2[1] + vs)/vs Does the above make sense (I followed the equation on p. 250 of the metafor reference manual)? Or is the H2 easily retrievable in another way? Thank you and all the best, Marlene
______________________________________________________ VUmc disclaimer : www.vumc.nl/disclaimer AMC disclaimer : www.amc.nl/disclaimer