Dear Nelly and all,
With respect to (only) the first question (sample size):
I think nothing is wrong, at least in principle, with a meta-analysis of
two studies. We analyze single studies, so why not combining two of
them? They may even include hundreds of patients.
Of course, it is impossible to obtain a decent estimate of the
between-study variance/heterogeneity from two or three studies. But if
the confidence intervals are overlapping, I don't see any reason to
mistrust the pooled effect estimate.
Best,
Gerta
Am 27.08.2020 um 16:07 schrieb ne gic:
Many thanks for the insights Wolfgang.
Apologies for my imprecise questions. By "agreed upon" & "what
conclusions/interpretations", I was thinking if there is a minimum sample
size whose pooled estimate can be considered somewhat reliable to produce
robust inferences e.g. inferences drawn from just 2 studies can be
drastically changed by the publication of a third study for instance -
it seems like there isn't. But I guess readers have to then check this
themselves to access how much weight they can place on the conclusions of
specific meta-analyses.
Again, I appreciate it!
Sincerely,
nelly
On Thu, Aug 27, 2020 at 3:43 PM Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:
Dear nelly,
See my responses below.
-----Original Message-----
From: R-sig-meta-analysis [mailto:
r-sig-meta-analysis-bounces at r-project.org]
On Behalf Of ne gic
Sent: Wednesday, 26 August, 2020 10:16
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] Sample size and continuity correction
Dear List,
I have general meta-analysis questions that are not
platform/software related.
*=======================*
*1. Issue of few included studies *
* =======================*
It seems common to see published meta-analyses with few studies e.g. :
(A). An analysis of only 2 studies.
(B). In another, subgroup analyses ending up with only one study in
the subgroups.
Nevertheless, they still end up providing a pooled estimate in their
respective forest plots.
So my question is, is there an agreed upon (or rule of thumb, or in
view) minimum number of studies below which meta-analysis becomes
unacceptable?
Agreed upon? Not that I am aware of. Some may want at least 5 studies
group or overall), some 10, others may be fine with if one group only
contains 1 or 2 studies.
What interpretations/conclusions can one really draw from such
That's a vague question, so I can't really answer this in general. Of
course, estimates will be imprecise when k is small (overall or within
groups).
*===================*
*2. Continuity correction *
* ===================*
In studies of rare events, zero events tend to occur and it seems
add a small value so that the zero is taken care of somehow.
If for instance, the inclusion of this small value via continuity
correction leads to differing results e.g. from non-significant results
when not using correction, to significant results when using it, what
make of that? Can we trust such results?
If this happens, then the p-value is probably fluctuating around 0.05
whatever cutoff is used for declaring results as significant). The
difference between p=.06 and p=.04 is (very very unlikely) to be
significant (Gelman & Stern, 2006). Or, to use the words of Rosnow and
Rosenthal (1989): "[...] surely, God loves the .06 nearly as much as the
.05".
Gelman, A., & Stern, H. (2006). The difference between "significant" and
"not significant" is not itself statistically significant. American
Statistician, 60(4), 328-331.
Rosnow, R.L. & Rosenthal, R. (1989). Statistical procedures and the
justification of knowledge in psychological science. American
If one instead opts to calculate a risk difference instead, and test
for significance, would this be a better solution (more reliable
to the continuity correction problem above?
If one is worried about the use of 'continuity corrections', then I
the more appropriate reaction is to use 'exact likelihood' methods
using (mixed-effects) logistic regression models or beta-binomial
instead of switching to risk differences (nothing wrong with the latter,
but risk differences are really a fudamentally different effect size
measure compared to risk/odds ratios).
Looking forward to hearing your views as diverse as they may be in
where there is no consensus.
Sincerely,
nelly
[[alternative HTML version deleted]]