[R-meta] 回复: terminologies of multilevel and multivariate model when accounting for correlated errors

Thu, Oct 6, 2022 4:56 PM

Great Wolfgang!
I really thank for your time on addressing my concern. I completely understand the two (differences and similirities) now after your explanation. I also want to be a person (like you) who is willing to spend himself/herself valuable time to help an unknown people (like me) when I think I am qualified to be. Very much appreciated!

Best,
Yefeng
________________________________
???: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer at maastrichtuniversity.nl>
????: 2022?10?6? 23:57
???: r-sig-meta-analysis at r-project.org <r-sig-meta-analysis at r-project.org>
??: Yefeng Yang <yefeng.yang1 at unsw.edu.au>
??: RE: terminologies of multilevel and multivariate model when accounting for correlated errors

Not 100% sure which two models you are comparing.

rma(yi, V, random = ~ 1 | study/obs)

and

rma(yi, V, random = ~ obs | study)

are identical (as long as the estimate of rho > 0) and neither assumes a 'constant within-study correlation (correlation of estimates or sampling errors within clusters)'. The correlations between the sampling errors are contained in V and those can be of any shape or form (as long as V is positive definite) and that goes for both models.

Both models assume that the correlations of the underlying true effects within studies follow a compound-symmetric structure. In the multilevel model, this happens as a function of the two random effects, in the multilevel parametrization this happens because struct="CS" by default.

But I think what you may be trying to get at is that the multivariate parameterization allows for more flexibility if we want to drop the assumption of compound symmetry. For example, in the Berkey et al. (1998) example, we can simply set struct="UN" to allow the amount of heterogeneity to differ for the two 'observations' within the studies (and if there would be more than two outcomes within studies, then each pair would be allowed to have a different correlation). This is indeed not something we can do with the multilevel formulation.

Best,
Wolfgang

________________________________
??????: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer at maastrichtuniversity.nl>
???????: 2022??10??4?? 18:54
?????: r-sig-meta-analysis at r-project.org <r-sig-meta-analysis at r-project.org>
????: Yefeng Yang <yefeng.yang1 at unsw.edu.au>
????: RE: terminologies of multilevel and multivariate model when accounting for
correlated errors

Dear Yefeng,

Please see below for my comments.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On
Behalf Of Yefeng Yang
Sent: Tuesday, 04 October, 2022 4:41
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] terminologies of multilevel and multivariate model when
accounting for correlated errors

Hi all (especially Wofgang & James),

My questions: I am confused about whether should we call a multilevel model with
a VCV matrix accounting for sampling variances still a multilevel model OR
should
we call it a multivariate model

I elaborate on my questions as follows:

For statistically dependent effect sizes, we usually have two[1] 'typical'
models
to deal with.

 1.  For dependence due to multilevel/nested structure (one study contributes
more than one effect size estimate), we usually use a multilevel model (with a
nested random effect structure) to account for the non-independence if there are
'overlapping individuals' (no correlated sampling errors).

If there are overlapping individuals (i.e., the same individuals are used in
computing multiple effect size estimates), then the sampling errors *are*
correlated, so I am a bit confused here.

So, let me assume for the moment that there are *no* overlapping individuals, but
a study can still yield multiple effect size estiamtes (e.g., for different
subgroups). Example of this are:

https://www.metafor-project.org/doku.php/analyses:konstantopoulos2011
https://www.metafor-project.org/doku.php/analyses:crede2010

The model typically used in a multilevel model with 'random = ~ 1 | study/obs' as
the random effects structure. However, note that we can reformulate this model
into a multivariate parameterization with 'random = ~ obs | study', which is
identical in fit (as long as the estimate of rho > 0).

So, already, I would say the terminology is a bit arbitrary, since we could call
this a multilevel or a multivariate model.

 2.  For dependence due to multivariate structure (one study contributes more
than one response variable or outcome), we usually use a multivariate model
(with
a correlated random effect structure) to account for the non-independence. Also,
we should use a variance-covariance matrix to account for the independent
sampling errors (either guessing within-study correlation or using formulas).

An example of this would be:

https://www.metafor-project.org/doku.php/analyses:berkey1998

This would be a 'classical' multivariate meta-analysis and I think most people
would call it that.

[1] robust variance estimation (RVE) is also a good approach to dealing with
dependent effect sizes in terms of estimating fixed effects (overall effect
intercept beta0 or moderator effect slope beta1).  The combination of the RVE
with either multilevel or multivariate is also an elegant approach. But RVE is
not the focus of my question.

However, sometimes we want to use the multilevel model to deal with all types of
independence. By doing so, we reformulate the multivariate structure of the data
as multilevel/nested data. I mean we: (1) use dummy codes to denote different
types of response variables/outcomes, (2) impute or calculate a VCV matrix, and
(3) fit a multilevel model.  Through (1) - (3) steps, I account for all types of
independence: the correlations between true outcomes and sampling errors. Not
100% sure, but this approach should work well.

An example along those lines would be (leaving aside the RVE stuff):

https://wviechtb.github.io/metadat/reference/dat.assink2016.html

or briefly:

dat <- dat.assink2016
V <- vcalc(vi, cluster=study, obs=esid, data=dat, rho=0.6)
rma.mv(yi, V, mods = ~ deltype, random = ~ 1 | study/esid, data=dat)

Again, this can be reformulated into:

rma.mv(yi, V, mods = ~ deltype, random = ~ esid | study, data=dat)

with identical fit. So, is this now a multilevel or multivariate model? I would
say either term is fine. But the terms are so broad anyway that they communicate
very little what was actually done, so either way, one should provide further
details (with respect to V and the random-effects structure).

So my question comes: I use a multilevel model but I also use a VCV matrix. What
will a multilevel model with a VCV be called? Still a multilevel model, but a
multilevel model assumes independent sampling errors (but we have a VCV in the
model). Should it be a multivariate model, but we did not account for the
correlated random effects only account for the correlated sampling errors? Hope
my question is clear.

Best,

Yefeng Yang PhD
Research Associate
UNSW, Sydney

[R-meta] 回复: terminologies of multilevel and multivariate model when accounting for correlated errors

Thread (5 messages)