Skip to content

[R-meta] forest plot and study-specific effect

7 messages · Wolfgang Viechtbauer, Dr. Gerta Rücker, Yefeng Yang +1 more

Original
Yefeng Yang
# 
Dear community,

I have a small question about the forest plot used in the meta-analysis.

The forest plots and their varieties are often used in meta-analysis papers to show the overall/grand mean, individual effect size estimates, and other relevant info depending on the software making them. We know the effect size estimates from individual studies are usually noisy or not very precise. But, why do meta-analysts prefer to report individual effect size estimates rather than the study-specific effects (which benefit from the shrinkage or borrowing of strength). Or, put differently, the developers of software that can make forest plots do not seem to provide the option of showing study-specific effects (I did not check carefully; some software or packages might provide this functionality). Is there any specific reason? Or, it is just a convention.

Best,
Yefeng
Wolfgang Viechtbauer
# 
Dear Yefeng,

Plotting the BLUPs has been done before. See Figure 3 in:

van Houwelingen, H. C., Arends, L. R., & Stijnen, T. (2002). Advanced methods in meta-analysis: Multivariate approach and meta-regression. Statistics in Medicine, 21(4), 589-624.

Actually, the forest plot shows the individual estimates, plus the BLUPs.

You can find a recreation of this here:

https://www.metafor-project.org/doku.php/analyses:vanhouwelingen2002

I think there are various reasons why the default is not to show the BLUPs. For example, the estimates are simply what was found in each of the invididual studies, while the BLUPs depend on what other studies are included in the analysis and the values also depend on the specifics of the modeling approach used. But showing both (as above) is definitely interesting.

Best,
Wolfgang
Dr. Gerta Rücker
# 
Dear Yefeng,

You may like to have a look at the supplement " traceplot-R-code.R" to the paper https://onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1693 (R?ver, Rindskopf, Friede) that shows how to obtain a joint forest plot with study-specific estimates and BLUPs with R package bayesmeta.

Best,
Gerta



UNIVERSIT?TSKLINIKUM FREIBURG
Institute for Medical Biometry and Statistics

Dr. Gerta R?cker
Guest Scientist

Stefan-Meier-Stra?e 26 ? 79104 Freiburg
gerta.ruecker at uniklinik-freiburg.de

https://www.uniklinik-freiburg.de/imbi-en/employees.html?imbiuser=ruecker

-----Urspr?ngliche Nachricht-----
Von: R-sig-meta-analysis <r-sig-meta-analysis-bounces at r-project.org> Im Auftrag von Viechtbauer, Wolfgang (NP) via R-sig-meta-analysis
Gesendet: Mittwoch, 5. Juni 2024 14:19
An: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis at r-project.org>
Cc: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer at maastrichtuniversity.nl>
Betreff: Re: [R-meta] forest plot and study-specific effect

Dear Yefeng,

Plotting the BLUPs has been done before. See Figure 3 in:

van Houwelingen, H. C., Arends, L. R., & Stijnen, T. (2002). Advanced methods in meta-analysis: Multivariate approach and meta-regression. Statistics in Medicine, 21(4), 589-624.

Actually, the forest plot shows the individual estimates, plus the BLUPs.

You can find a recreation of this here:

https://www.metafor-project.org/doku.php/analyses:vanhouwelingen2002

I think there are various reasons why the default is not to show the BLUPs. For example, the estimates are simply what was found in each of the invididual studies, while the BLUPs depend on what other studies are included in the analysis and the values also depend on the specifics of the modeling approach used. But showing both (as above) is definitely interesting.

Best,
Wolfgang
_______________________________________________
R-sig-meta-analysis mailing list @ R-sig-meta-analysis at r-project.org
To manage your subscription to this mailing list, go to:
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
Röver, Christian
# 
Dear Yefeng,

the forest plots in the "bayesmeta" package by default show the
shrinkage estimates / BLUPs along with the plain data; see e.g. here:
  https://cran.r-project.org/package=bayesmeta
  https://doi.org/10.18637/jss.v093.i06  (Fig.2)
(bayesmeta's forest plots are based on the "forestplot" package).

I often find this helpful to illustrate the amount of borrowing-of-
information / mutual support between studies, depending on the amount
of heterogeneity. It may in particular also be an interesting option in
cases where the "plain data estimates" may not be readily available,
e.g., when you are looking at a binomial likelihood and some studies
have zero event counts.

Cheers,

Christian


On Wed, 2024-06-05 at 16:54 +0000, Dr. Gerta R?cker via R-sig-meta-
analysis wrote:
Yefeng Yang
# 
Dear Wolfgang, Gerta, and Christian,

Thank you very much for your thoughts.
The recommended sources (metafor example, papers, and bayesmeta package) are quite useful.

All the best,
Yefeng
Yefeng Yang
# 
Dear Christian,

I had a read of your introductive paper about bayesmeta.

A quick question to ask. Under the subsection 2.8. Shrinkage estimates of study-specific means, the formula for computing study-specific effects is presented. I am wondering whether it makes sense to plug-in tau^2_i (tau^2 for each study) rather than the mean/median/mode to the formula, given that each study might not have the same heterogeneity.


All the best,
Yefeng
11 days later
Röver, Christian
# 
Hi Yefeng,

note that the formulas in Sec. 2.8 relate to the *conditional*
distribution of the theta_j parameters (conditional on tau), which is a
normal distribution, characterized by mean and variance. In order to
arrive at the eventual marginal posterior, one still needs to integrate
over the heterogeneity (tau) posterior, yielding a non-normal posterior
distribution. In particular, there is no mean/median/mode plugged in.
If you did that, it may in some cases still be a reasonable
approximation; this is commonly done in an "empirical Bayes" type
analysis. (The "bayesmeta()" function will return the "proper" marginal
posterior for you.)

But your actual point relates to the original model specification (the
normal-normal hierarchical model) from Sec. 2.1 -- of course, one can
think of alternatives here, with more than a single heterogeneity
parameter. The simplest extension may e.g. be a model where you have
two groups of studies estimating the same parameter. Suppose that one
group reports unadjusted estimates, while the other group has adjusted
for some covariable. Then you might expect different amounts of
heterogeneity for the two groups. This could be implemented as special
case of a "location-scale model" as described by Viechtbauer and
Lopesz-Lopez (2022) here: https://doi.org/10.1002/jrsm.1562

This kind of model of course is only really helpful as long as you have
several studies in each group; with few studies per subgroup (or even
single studies in a group, or a separate heterogeneity parameter for
certain individual studies) it gets hard to estimate the heterogeneity,
as you are effectively trying to estimate a variance based on a single
(or few) observation(s).

If you still think the assumption of a single common normal variance
parameter is too restrictive, you may also consider using as model as
implemented in the "metaplus" package (
https://doi.org/10.32614/CRAN.package.metaplus), where the random
effect follows a non-normal distribution. For example, if you think
there are two groups of studies, one more and one less heterogeneous
(but you don't know which study belongs to which group), then using a
two-component normal mixture may do the trick. Or if you think the
(normal) heterogeneity variance may not be identical for all studies,
but may itself be somewhat heterogeneous, this may marginally lead to a
Student-t distributed random effect, where the degrees-of-freedom
parameter regulates the variability of the heterogeneity parameters. I
would however expect that you need quite a lot of data (i.e., studies)
to estimate these more complex models with reasonable precision.
However, if you think these models describe reality more accurately
this may be the way to go; and a Bayesian approach with informative
priors might help in case of fewer studies.

Cheers,

Christian
On Thu, 2024-06-06 at 08:14 +0000, Yefeng Yang wrote: