Dear metafor users,
I downloaded the data set (i.e., p154-dataset.csv) from Assinka and Wibbelink (2016) available from http://www.tqmp.org/RegularArticles/vol12-3/p154/p154.pdf
The data set contains 100 studies and I compared I^2 statistics from the model with no moderator (overall) and the one with 2 significant moderators (multiplemoderator).
1) Model overall (Listing 5 in the publication) and I^2 statistics:
overall <- rma.mv(y, v, random = list(~ 1 | effectsizeID, ~ 1 | studyID), tdist=TRUE, data=dataset)
# Calculation of I^2 statistic
W <- diag(1/overall$vi)
X <- model.matrix(overall)
P <- W - W %*% X %*% solve(t(X) %*% W %*% X) %*% t(X) %*% W
100 * sum(overall$sigma2) / # I^2 statistic
# Separation of total variance by cluster
100 * overall$sigma2 / (sum(overall$sigma2) + (overall$k-overall$p)/sum(diag(P)))
[1] 34.754 58.304
2) Model multiplemoderator (Listing 5 in the publication) and I^2 statistics:
multiplemoderator <- rma.mv(y, v, mods = ~ pyear + typeovert + typecovert,
random =list(~ 1 | effectsizeID, ~ 1 | studyID),
tdist=TRUE, data=dataset)
# Calculation of I^2 statistic
W <- diag(1/multiplemoderator$vi)
X <- model.matrix(multiplemoderator)
P <- W - W %*% X %*% solve(t(X) %*% W %*% X) %*% t(X) %*% W
100 * sum(multiplemoderator$sigma2) / # I^2 statistic
# Separation of total variance by cluster
100 * multiplemoderator$sigma2 / (sum(multiplemoderator$sigma2) + (multiplemoderator$k-multiplemoderator$p)/sum(diag(P)))
[1] 33.079 57.913
Questions:
The sampling variance is small at 7% to 11% in these models.
What is the correct interpretation of I^2 results? And for the slight reduction in sampling variance despite the addition of significant moderators?
I get similar results from my own data set with a large number of studies despite the addition of several significant moderators in the model.
Thanks in advance,
Roger :)
Roger Martineau, mv Ph.D.
Centre de recherche et de d?veloppement
sur le bovin laitier et le porc
Agriculture et agroalimentaire Canada/Agriculture and Agri-Food Canada
T?l?phone/Telephone: 819-780-7319
T?l?copieur/Facsimile: 819-564-5507
2000, Rue Coll?ge / 2000, College Street
Sherbrooke (Qu?bec) J1M 0C8
Canada
roger.martineau at agr.gc.ca<mailto:roger.martineau at agr.gc.ca>