[R-meta] Appending a risk-of-bias traffic-light plot to a 'three-level' forest plot
Dear Josh, See below for my responses. Best, Wolfgang
-----Original Message----- From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Joshua Bernal Sent: Friday, 18 February, 2022 3:07 To: r-sig-meta-analysis at r-project.org Subject: [R-meta] Appending a risk-of-bias traffic-light plot to a 'three-level' forest plot Greetings Dr. Viechtbauer and members, I ran a three-level meta-analysis with rma.mv (due to multiple measures of a construct within-studies). I seek help on how to append a risk-of-bias traffic-light plot to an extension of the three-level forest plot, please find the query details below... Goal A (metafor): Create the three-level forest plot with each study's "aggregated" effect estimate and CI. (Source A: stat.ethz.ch/pipermail/r-sig-meta-analysis/2019-February/001423.html) Goal A Issues: 1) Source A presents 2 ways to obtain "aggregated" effect sizes, with the second way getting "exactly the same results as those from the full model". Because the present rma.mv analysis uses the t-distribution (test = "t"), the p-values and summary CIs obtained are slightly different/larger. Is it possible to get the exact same results as when test = "t" for the rma.mv model? To see what this looks like, please find the metafor reprex below.
I would suggest to do the following:
1) Use the 'contain' method for computing the dfs:
res <- rma.mv(yi, vi, random = ~ 1 | author/outcome, data = dat, test = "t", dfs = "contain",
method = "REML", slab = author)
2) Then use test="t" for rma():
rma(yi, vi, data=sav, test="t")
Then the results match up exactly.
2) After running the reprex, you will see that Study 6 contributes one effect size only, its CI from the "standard" rma.mv forest plot (where all effect sizes in each study are shown) vs its CI from the "aggregated" forest plot are different. If the argument "tau2=res$sigma2[2]" is omitted from the code of the "aggregated" forest plot, then the obtained summary CI for Study 6 then matches with that of the "standard" plot. Which would be considered the correct approach in this case?
The first. If you want to show a forest plot of the aggregated data, then the aggregation involves adding the within-study variance (as you have done) and hence the CI will be wider also for study 6.
metafor reprex:
###
library(metafor)
dat <- structure(list(author = c("Study 1", "Study 1", "Study 1", "Study 1",
"Study 1", "Study 2", "Study 2", "Study 2", "Study 2", "Study 3",
"Study 3", "Study 4", "Study 4", "Study 5", "Study 5", "Study 6"
), study = c(1, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 4, 4, 5, 5, 6),
outcome = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14,
15, 16), yi = c(0.52, 0.256, 0.5003, 0.3131, -0.1019, 0.3591,
0.0341, 0.0357, 0.3942, -0.1503, 0.2671, 0.131, 0.1664, 1.1761,
0.5799, 0.406), vi = c(0.00926, 0.00199, 0.01171, 0.0171,
0.00524, 0.0712, 0.0971, 0.0823, 0.0912, 0.1204, 0.3536,
0.0655, 0.0607, 0.9571, 0.6617, 0.5139)), row.names = c(NA,
16L), class = "data.frame")
res <- rma.mv(yi, vi, random = ~ 1 | author/outcome, data = dat, test
= "t", method = "REML", slab = author)
res
forest(res)
sav <- lapply(split(dat, dat$author), function(x) {
res <- rma(yi, vi, data=x, tau2=res$sigma2[2])
return(c(coef(res), vcov(res)))
})
sav <- data.frame(do.call(rbind, sav))
names(sav) <- c("yi", "vi")
forest(sav$yi, sav$vi, slab=rownames(sav))
rma(yi, vi, data=sav)
res
###
Goal B (robvis, development version 0.3.0.900):
Create a risk-of-bias traffic-light plot using the "Generic" (or aka
"ROB1") template.
(Source B: https://github.com/mcguinlu/robvis ("Generic") OR
cran.r-project.org/web/packages/robvis/vignettes/Introduction_to_robvis.html#rob1
("ROB1"))
Goal B Issues:
After running the robvis reprex below, you will see that an extra
judgment label ("Critical" or dark red circle with exclamation mark)
is automatically added/cannot be removed (although only 5 judgment
labels are relevant in the dataset, I am forced to specify 6 judgement
labels). I am aware there is an argument "judgement_levels" under
development, but do not know if it is relevant and if so how to take
it from here...
Can't help with this.
robvis reprex:
###
library(robvis)
rp <- structure(list(Study = c("Study 1", "Study 2", "Study 3", "Study 4",
"Study 5", "Study 6"), Bias.due.to.confounding. = c("Not applicable",
"Not applicable", "Not applicable", "Some concerns", "Low", "No information"
), Bias.in.selection.of.participants.into.the.study. = c("Not applicable",
"Not applicable", "Not applicable", "Some concerns", "Low", "Low"
), Bias.in.classification.of.interventions. = c("Not applicable",
"Not applicable", "Not applicable", "Some concerns", "Low", "Low"
), Bias.arising.from.the.randomization.process. = c("Some concerns",
"Low", "Low", "Not applicable", "Not applicable", "Not applicable"
), Bias.due.to.deviations.from.intended.intervention. = c("Low",
"Some concerns", "Low", "Low", "Some concerns", "Low"),
Bias.due.to.missing.outcome.data. = c("Some concerns",
"Low", "No information", "Some concerns", "Low", "No information"
), Bias.in.measurement.of.the.outcome. = c("Some concerns", "Low",
"Some concerns", "Some concerns", "Low", "Some concerns"),
Bias.due.to.selection.of.the.reported.result. = c("Some concerns",
"Low", "High", "Some concerns", "Low", "High"), Overall = c("Some concerns",
"Low", "High", "Some concerns", "Low", "High")), row.names = c(NA,
6L), class = "data.frame")
bor <- rob_traffic_light(rp, tool="Generic", colour="cochrane",
psize=10, overall=TRUE, judgement_labels=c("","High","Some
concerns","Low","No information","Not applicable"))
bor
###
Goal C:
Append the risk-of-bias traffic-light plot (from Goal B) to the forest
plot (from Goal A).
(Source C: mcguinlu.github.io/robvis/articles/metafor.html#appending-risk-of-
bias-plots-to-forest-plots-1)
Goal C Issues:
Because the present analysis considers the bias domains of both
randomized and non-randomized controlled intervention studies and risk
of bias was respectively assessed by ROB2 and ROBINS-I, the "Generic"
template is used because it allows modifying the number of domains so
as to include the relevant domains of both tools in the traffic-light
plot. However, according to "rob_tools(forest = TRUE)", robvis
currently supports only "ROB2" and "ROBINS-I" templates for the
"rob_append_to_forest()" function.
Again, can't really help with robvis. As an alternative, you could also consider creating the traffic light part of the plot yourself. Then you have full control over what is shown and how. An example where I did this can be found here: https://www.metafor-project.org/doku.php/plots:forest_plot_revman
Other sub-goals: - Modify text of header from "Estimate [95% CI]" to "SMD [95% CI]".
In forest(), see argument 'header'.
- Modify text of x-axis label from "Observed Outcome" to "Standardized Mean Difference".
In forest(), see argument 'xlab'.
- To the right of "SMD [95% CI]", add a column with the header "No. of effect sizes" to display the number of effect sizes within each study.
In forest(), you can use the 'textpos' argument to create space to the right of the annotations and then add this column either via the 'ilab' argument or with text(). In fact, the link above demonstrates this.
Many thanks in advance for your help... Sincerely, Josh