[R-meta] Meta-analysis with observational and experimental studies

Tue, Nov 7, 2023 1:31 AM

Dear Karen,

I suspect this thread isn't going to lead to a whole lot of useful responses (but would love to be proven wrong). The reason why I say this is that you seem to be at the stage of a meta-analysis where the goal is to figure out how to structure the data, what kind of effect sizes can be computed, and what kind of analyses can be conducted, given the goals and purposes of the meta-analysis itself, under the constraints of the types of studies that have been conducted and the information reported therein. In my experience, this is the most difficult task of a meta-analysis with complex data structures. This step is best carried out in person where we can look together at the data, some example studies, and some possible ways of structuring the data, using a white board or a whole lot of scratch paper.

But briefly: There is no absolute rule on whether one can meta-analyze observational and experimental research together or not. Whether this makes sense or not is highly context dependent. Beyond this, I cannot comment on any approaches without a *lot* more details. But again, I suspect a mailing list like this is not a replacement for sitting down in person when it comes to addressing such general questions.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis <r-sig-meta-analysis-bounces at r-project.org> On Behalf
Of PARRA DE LA ROSA, KAREN via R-sig-meta-analysis
Sent: Sunday, November 5, 2023 13:48
To: Michael Dewey <lists at dewey.myzen.co.uk>
Cc: PARRA DE LA ROSA, KAREN <karen.parra01 at estudiant.upf.edu>; R Special
Interest Group for Meta-Analysis <r-sig-meta-analysis at r-project.org>
Subject: Re: [R-meta] Meta-analysis with observational and experimental studies

Dear Michael,

Many thanks for your reply. I am interested in multiple groups in each
trial. What you are suggesting is to use the raw mean per group and
then transform it into r (which is the metric I am using) instead of
calculating the contrast between groups?

Best,

Karen

El dom, 5 nov 2023 a las 12:49, Michael Dewey
(<lists at dewey.myzen.co.uk>) escribi?:

Dear Karen

I will leave it to others to coment on model choice but one thing which
occurs to me is that if you just use one arm in a trial does that not
convert it into an observational study?

Michael

On 05/11/2023 10:15, PARRA DE LA ROSA, KAREN via R-sig-meta-analysis wrote:

Hi everyone,

I am conducting a systematic review and meta-analysis on the impact of
different types of appearance-focused social media activities on body
image. I wanted to test a novel categorization of these types of
social media activities so we can better understand if the shared
characteristics of different activities could explain their
effect/relationship with body image. On the other hand, I am also
interested in having an estimate per activity. The dataset has 130
studies, half observational (cross-sectional, longitudinal, experience
sampling), and half experimental studies. The vast majority of them
contribute more than one effect size because of different activities
and multiple outcomes. This is my first meta-analysis and also my
first time using R so I have many questions regarding how to specify
the model.

I have read that observational and experimental research cannot be
synthesized in the same meta-analysis as they are answering different
questions. So I was running two different analyses for each type of
study. As for observational studies I fitted with metafor and
clubsandwich a correlated and hierarchical model with robust variance
estimation to account for the multiple dependencies among effect
sizes. So effect sizes are nested within studies. However, the problem
I am facing is with experimental studies. I have organized them in a
different file with the mean, sd, and n per treatment condition. I am
not interested in the comparison with control groups, rather I am
interested in understanding if, for instance, exposure to body ideals
is better/worse than posting this type of content. I have several
comparisons like the one in the example. I thought that a network
meta-analysis in netmeta could be the best solution but as far as I
know, this package does not allow for moderator analysis, so could not
be able to run a subgroup analysis with the categories of activities.
I do not know what the best approach to analyze this data. Also, I
have never seen in my field a paper with two different types of
meta-analysis: a three-level for observational and an NMA for
experimental.

Hope someone on the mailing list could help me with some of this doubt.

Thanks in advance for your help and guidance.

Best,

Karen

[R-meta] Meta-analysis with observational and experimental studies

Thread (4 messages)