[R-meta] Studies with more than one control group

Dear James,

I'm coming back to this after a while (preparing the data). A quick
follow-up. So, you mentioned that if I have several studies that have used
more than 1 control group (in my data up to 2), I can possibly add a
random-effect (controlID) to capture any heterogeneity in the effect sizes
across control groups nested within studies.

My question is that adding a controlID random-effect (a binary indicator:
1 or 2) would also mean that we intend to generalize beyond the possible
number of control groups that a study can employ (for my data beyond 2
control groups)?

Thank you,
Jack

On Thu, Jun 24, 2021 at 4:52 PM Jack Solomon <kj.jsolomon at gmail.com>
wrote:

Thank you very much for the clarification. That makes perfect sense.

Jack

On Thu, Jun 24, 2021 at 4:44 PM James Pustejovsky <jepusto at gmail.com>
wrote:

The random effect for controlID is capturing any heterogeneity in the
effect sizes across control groups nested within studies, *above and beyond
heterogeneity explained by covariates.* Thus, if you include a covariate to
distinguish among types of control groups, and the differences between
types of control groups are consistent across studies, then the covariate
might explain all (or nearly all) of the variation at that level, which
would obviate the purpose of including the random effect at that level.

On Thu, Jun 24, 2021 at 9:56 AM Jack Solomon <kj.jsolomon at gmail.com>
wrote:

Thank you James. On my question 3, I was implicitly referring to my
previous question (a previous post titled: Studies with independent
samples) regarding the fact that if I decide to drop 'sampleID', then I
need to change the coding of the 'studyID' column (i.e., then, each sample
should be coded as an independent study). So, in my question 3, I really
was asking that in the case of 'controlID', removing it doesn't require
changing the coding of any other columns in my data.

Regarding adding 'controlID' as a random effect, you said: "... an
additional random effect for controlID will depend on how many studies
include multiple control groups and whether the model includes a covariate
to distinguish among types of control groups (e.g., business-as-usual
versus waitlist versus active control group)."

I understand that the number of studies with multiple control groups is
important in whether to add a random effect or not. But why having "a
covariate to distinguish among types of control groups" is important in
whether to add a random effect or not?

Thanks, Jack

On Thu, Jun 24, 2021 at 9:17 AM James Pustejovsky <jepusto at gmail.com>
wrote:

Hi Jack,

Responses inline below.

James

I have come across a couple of primary studies in my meta-analytic
pool
that have used two comparison/control groups (as the definition of
'control' has been debated in the literature I'm meta-analyzing).

(1) Given that, should I create an additional column ('control') to
distinguish between effect sizes (SMDs in this case) that have been
obtained by comparing the treated groups to control 1 vs. control 2
(see
below)?

Yes. Along the same lines as my response to your earlier question, it
seems prudent to include ID variables like this in order to describe the
structure of the included studies.

(2) If yes, then, does the addition of a 'control' column call for the
addition of a random effect for 'control' of the form:  "~ |
studyID/controlID" (to be empirically tested)?

I expect you will find differences of opinion here. Pragmatically, the
feasibility of estimating a model with an additional random effect for
controlID will depend on how many studies include multiple control groups
and whether the model includes a covariate to distinguish among types of
control groups (e.g., business-as-usual versus waitlist versus active
control group).

At a conceptual level, omitting random effects for controlID leads to
essentially the same results as averaging the ES across both control
groups. If averaging like this makes conceptual sense, then omitting the
random effects might be reasonable.

(3) If I later decide to drop controlID from my dataset, I think I can
still keep all effect sizes from both control groups intact without
any
changes to my coding scheme, right?

I don't understand what you're concern is here. Why not just keep
controlID in your dataset as a descriptor, even if it doesn't get used in
the model?