[R-meta] meta-analysis vs. re-analysis

Elli,

These are very interesting questions. You will find relevant references
using the terms ?individual participant data meta-analysis? (e.g. Riley,
Lambert, & Abo-Zaid, 2010, https://www.jstor.org/stable/pdf/25674217.pdf
<https://www.jstor.org/stable/pdf/25674217.pdf?casa_token=rRC6UCpidkoAAAAA:Tqo4pIznk-dIPn521YGrvI9Kg3fMo-7eDCbbbMnjMVMxiUsXOrZ6XyY22wRLWoKaY5hONfBWaBGFTark_zXp8EKb6qkTZKU73XQbocMlULNikKA-l8vN>)
and ?integrative data analysis? (Curran & Hussong, 2009,
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2777640/). A recent review
is by Debray and colleagues (2015,
https://onlinelibrary.wiley.com/doi/abs/10.1002/jrsm.1160).

Regarding whether you need to conduct sensitivity analysis or consider the
potential for publication bias, I would argue that these remain important
even though you have analyzed the individual-level data. Just having access
to the IPD does not alleviate the possibility that the samples you have
identified might not represent the full body of relevant research conducted
on your topic. One paper that discusses these issues is Ahmed, Sutton, &
Riley (2012; https://doi.org/10.1136/bmj.d7762).

James

On Sep 7, 2019, at 4:41 PM, Elli J. Theobald <ellij at uw.edu> wrote:

Greetings!

I am wondering about the differences (and some citations to support the
logic) between meta-analyses and re-analyses. We conducted a systematic
review of the literature, then of the papers we decided to include in our
study, we contacted the authors and requested their raw data. (I think that
the reason for this decision is irrelevant to my question but I would be
happy to explain if that is helpful.)

With their raw data, we answered the question(s) we were interested in by
fitting hierarchical Bayesian regression models (controlling for study and
other clustering/non-independence elements within each study. We were
interested in student performance within different types of classrooms.)

My intuition is that:
1) This is a re-analysis of the data, not a meta-analysis (because there
was nothing meta about it!)
2) We do not need to or show any of the typical quality
assurance/sensitivity analyses, like fail safe number, funnel plots, etc. I
am not even entirely sure how we would do this given that our unit of
observation is finer-grain than study. (We re-analyzed the student-level
data the original papers published.) We have included a histogram of the
means from each study to show that it is roughly normal in shape, to show
that we don't have crazy sampling.

Can anyone weigh in? Am I distinguishing re-analysis from meta-analysis
appropriately (or is the semantics debate unwarranted)? Can you point me to
a good citation(s) that distinguish re-analyses from meta-analyses? And is
my intuition correct that we don't need to provide additional sensitivity
analyses?

Thanks so much for your help!
Cheers.
Elli


--
<http://www.biology.washington.edu/users/elli-jenkins>Elli J. Theobald,
PhD
Research Scientist
Biology Education Research Group
Department of Biology
University of Washington, Seattle
https://sites.google.com/site/ellijtheobald
<https://sites.google.com/site/ellijtheobald/home>

   [[alternative HTML version deleted]]