[R-meta] rule of thumb miminum number of studies per factor level meta-regression

Dear Lena,

Just for the record, the '10 studies per covariate' rule comes from here:

https://training.cochrane.org/handbook/current/chapter-10#section-10-11-5-1

where it says:

"It is very unlikely that an investigation of heterogeneity will produce useful findings unless there is a substantial number of studies. Typical advice for undertaking simple regression analyses: that at least ten observations (i.e. ten studies in a meta-analysis) should be available for each characteristic modelled. However, even this will be too few when the covariates are unevenly distributed across studies."

I have no idea where the 10 per covariate rule comes from (there is also no reference in the Cochrane Handbook) and I am not aware of any empirical support for it. I suspect it was just taken over from similar rules that have been formulated in other contexts (e.g., regression models with primary data, prediction models, factor analysis) where these rules have often been formulated without much, if any, empirical support.

Given what it says in the Cochrane Handbook, one could read this to imply that at least 10 studies per covariate are needed to 'produce useful findings'. Without a definition of 'useful findings', I don't even know how to evaluate whether such a rule is sensible or not.

I am not trying to rag on the Cochrane Handbook. The question about 'k per moderator' (or k in general for a meta-analysis) is one of the questions that *always* comes up in any course on meta-analysis I teach. It is a good question and I have no good answer for it, except to mention that such rules exist (e.g., '10 per covariate'), but that they lack empirical support.

Analogously, I am not aware of any evidence-based guidelines with respect to your 'k per level' question.

So, in the end, I am doing again the same thing as I always do when I get this question, which is to provide no good answer.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On
Behalf Of Lena Pollerhoff
Sent: Monday, 28 March, 2022 10:40
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] rule of thumb miminum number of studies per factor level meta-
regression

Dear list member,

I am conducting meta-regressions in metafor at the moment and have a short
question regarding rule of thumbs with respect to categorical predictors in meta-
regression. While we are aware of one rule of thumb that meta-regressions should
not be considered for fewer than ten studies per covariate (e.g., Cochrane
Handbook), we were wondering whether such a rule of thumb also exists with
respect to the minimum number of studies per factor level of a categorical
variable?

In my case, I am conducting meta-regressions, where the number of studies per
factor level are sometimes unevenly distributed: For example, k = 22, and I have
one categorical predictor with three factor levels, with the first one
represented by only one study, the second one by three studies, and the third one
including 18 studies.

Thanks in advance and have a nice day!
Lena Pollerhoff