[R-meta] Sample size and continuity correction

Dear Nelly,

you may need to distinguish between frequentist and Bayesian methods
here.

Firstly, you may wonder how "representative" a small sample can
possibly be of some general population, however, when you think about
it, this is not necessarily an issue tied to small samples -- you could
also think of large samples that are not representative, e.g., due to
selection biases.

Secondly, small sample sizes (small numbers of studies or few events
within a study) may lead to "technical" difficulties for the meta-
analysis methods. Consider for example the normal approximation that is
often utilized in a normal model; this tends to break down e.g. if you
are looking at a log-OR endpoint and you only have one, two or no
events in one of the study arms. Continuity corrections then may help,
but only to a certain degree. Such issues are discussed e.g. by Jackson
and White (2018; https://doi.org/10.1002/bimj.201800071).

You can then substitute the Normal approximation by a more accurate
model (e.g., a Binomial likelihood; see e.g. the proposals discussed by
Seide et al. (2018; https://doi.org/10.1186/s12874-018-0618-3)).
However, many methods may still perform unsatisfactorily for few
studies or few events, essentially because they often rely on many-
study and/or many-event asymptotics.

This is where frequentist and Bayesian methods may behave somewhat
differently. Bayesian methods generally behave reasonably for any study
number or size, however, the asymptotics issue does not completely go
away. For many studes and many events, the prior information that is
formally included in the model tends to make little difference; but the
fewer studies or events you have, the more important the prior
assumptions will become. It hence crucial to convincingly motivate the
prior assumptions you make. A fully Bayesian approach for few studies
and events (based on a binomial model) is described e.g. by G?nhan et
al. (2020; https://doi.org/10.1002/jrsm.1370). Within the common normal
model, you usually first of all have to worry about prior specification
for the heterogeneity parameter; we have recently summarized some
guidance here: https://arxiv.org/abs/2007.08352 .

Cheers,

Christian

Dear List,

I have general meta-analysis questions that are not
platform/software related.

*=======================*
*1. Issue of few included studies *
* =======================*
It seems common to see published meta-analyses with few studies e.g.
:

(A). An analysis of only 2 studies.
(B). In another, subgroup analyses ending up with only one study in
one of
the subgroups.

Nevertheless, they still end up providing a pooled estimate in their
respective forest plots.

So my question is, is there an agreed upon (or rule of thumb, or in
your
view) minimum number of studies below which meta-analysis becomes
unacceptable?

What interpretations/conclusions can one really draw from such
analyses?

*===================*
*2. Continuity correction *
* ===================*

In studies of rare events, zero events tend to occur and it seems
common to
add a small value so that the zero is taken care of somehow.

If for instance, the inclusion of this small value via continuity
correction leads to differing results e.g. from non-significant
results
when not using correction, to significant results when using it, what
does
make of that? Can we trust such results?

If one instead opts to calculate a risk difference instead, and test
that
for significance, would this be a better solution (more reliable
result?)
to the continuity correction problem above?


Looking forward to hearing your views as diverse as they may be in
cases
where there is no consensus.

Sincerely,
nelly

	[[alternative HTML version deleted]]

_______________________________________________
R-sig-meta-analysis mailing list
R-sig-meta-analysis at r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis