[R-meta] IPD meta analysis / complex survey design

Hi Corentin,

I did not mean to suggest that one should run several different analyses
on a single dataset. That would indeed place too much of a burden on the
authors of the individual studies.

My suggestion is really about this part:

Our question was whether - within the same meta-analysis - we could
"safely" include effect sizes estimated by a standard logistic regression

(when

data have a regular structure) +  effect sizes estimated by the svyglm

function

(when the data have a complex structure).

I cannot tell you if is safe or not. But what you can always do is combine
these different types in a single analysis and then check if there are
systematic differences between these two types of effect sizes. If there
are no systematic differences, then this is (empirical) evidence that
combining them is in some sense an acceptable thing to do.

This approach is similar to checking if effect sizes extracted from
published articles are systematically different from those extracted from
unpublished sources in a meta-analysis. If there are systematic
differences, we need to think about what the reason for the difference may
be. If not, then this is one less thing to worry about.

Best,
Wolfgang

-----Original Message-----
From: GOSLING Corentin [mailto:corentin.gosling at gmail.com]
Sent: Friday, 05 March, 2021 10:33
To: Viechtbauer, Wolfgang (SP)
Cc: r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] IPD meta analysis / complex survey design

Dear Prof Viechtbauer,

Thank you very much for your reply!

Sorry, my question was a bit misleading. In line with your suggestion,

our aim is

to avoid merging ?marginal ?coefficients and ?conditional? coefficients

by using

only the svyglm function as soon as the data has a complex structure

(clustering

and/or weighting, etc...).

You are entirely right, in situations with clustering only, we could

compare 3

approaches : (i) select only 1 individual per cluster and use glm

function, or

keep clustering and use (ii) glmer function or (iii) svyglm function.

However,

we are a bit reluctant to make these comparisons for two reasons.

First, as soon

as data have a more complex structure (e.g. sampling weights), the only

approach

allowing to take this into account is the svyglm function. This makes

comparisons

a bit strange, as in our examples, since one analysis is taking account

of some

specificity of the design while the others are not. Second, from a

practical point

of view, the burden on authors will become even more complicated as the

time

required for analysis is already sometimes quite long (in particular

because of

several multiple imputation models). We are concerned that the

multiplication of

tests may sometimes make the analysis time so long that it may discourage

some

authors from participating.

Our question was whether - within the same meta-analysis - we could
"safely" include effect sizes estimated by a standard logistic regression

(when

data have a regular structure) +  effect sizes estimated by the svyglm

function

(when the data have a complex structure). By safely, I mean without

having to

compare the results of the svyglm function to other functions (such as

glm or

glmer) when data have a complex structure.

If this is not possible, a more anecdotal question was whether it is

possible to

"safely" include  effect sizes estimated by a  standard logistic

regression (when

data have a regular structure) + effect sizes estimated by the glmer

function

(when data have clustering).

Thank you so much for your help!

Best wishes
Corentin Gosling

Le ven. 5 mars 2021 ? 09:32, Viechtbauer, Wolfgang (SP)
<wolfgang.viechtbauer at maastrichtuniversity.nl> a ?crit :
Dear Corentin,

I cannot answer your question directly, that is, to what extent those

results are

comparable to each other, although if svyglm() gives 'marginal'

(population

averaged) coefficients in the sense of what a GEE model would do, then

one could

argue that those should not be combined with 'conditional' coefficients

that

glmer() provides (searching for combinations of terms like "GEE, marginal,
population averaged, logistic mixed-effects, conditional,

subject-specific" should

turn up relevant discussions / papers).

But leaving this aside, one could also just approach this issue entirely
empirically, that is, simply code the type of analysis / type of

coefficient for

each study and examine in a moderator analysis whether there are

systematic

differences between the different types.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:

r-sig-meta-analysis-bounces at r-project.org] On

Behalf Of GOSLING Corentin
Sent: Thursday, 04 March, 2021 11:29
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] IPD meta analysis / complex survey design

Dear all

I come back to you about the IPD meta-analysis we are conducting to

explore

the effect of month of birth on the persistence of ADHD. I had already
asked for your help a few months ago when I was writing the protocol. We
have since completed our systematic review and started to include data

from

different cohorts. As the month of birth is sensitive data, we do not ask
the authors to send us the raw data: we have constructed an R-script that
we send to the authors and which performs the analyses automatically and
shares the anonymised results. We then carry out a classic two-stage
meta-analysis based on summary results.

We are facing a new challenge that we did not anticipate. Several studies
involve complex survey design. Some studies have clusters (e.g., twin
cohorts or assessments of several regular siblings per family), while
others have even more complex sampling (and include for example sampling
weights, stratum or finite population correction (fpc)). Some studies
include both (clusters + stratum/weights/fpc).

To analyse the data with clustering, naturally we thought of using mixed
models via the glmer function of lme4 (our VD is binary: ADHD persistence
yes/no). However, lme4 does not allow to handle - for the moment -

sampling

weights or stratifications. Therefore, for all data with clustering

and/or

weights and/or stratum and/or fpc, our idea was to use only the svyglm
function of the survey package in order to have a coherent group of
analyses (we know that the glmer and svyglm functions do not use the same
coefficients (marginals vs. conditionals)).

Our question is the following: can we group within the same meta-analysis
coefficients that come from standard logistic regressions and

coefficients

that come from generalised mixed models fitted using glmer or generalised
linear models adapted to complex designs fitted using svyglm?

To support our question, we performed some tests on a dataset including
clusters and sampling weights. Here are the results :

[...]

As you can see, the results are almost the same from the models, except
when we take into account sampling weights. I hope that our problem is
clearly exposed

Thank you very much in advance for your help!

Corentin J Gosling