[R-meta] Overlapping CIs with significant difference among subgroups

Tue, Oct 6, 2020 12:17 PM

Dear Rafael,

The SEs of the predicted average outcomes for the various levels can be quite different than the SEs of the difference between levels.

You can get the predicted average outcome for all four levels with:

pred.r = predict(res, transf=transf.ztor, digits=3, newmods = rbind(c(0,0,0), c(1,0,0), c(0,1,0), c(1,1,1)))
pred.r

And then you can plot them with:

forest(pred.r$pred, ci.lb=pred.r$ci.lb, ci.ub=pred.r$ci.ub, slab=c("Without grouping","Temporal grouping","Spatial grouping", "Both"))

Indeed, CIs are wide and overlap. But let's now compute the predicted average difference between levels when compared against the "Without grouping" level:

pred.r = predict(res, transf=transf.ztor, digits=3, newmods = rbind(c(1,0,0) - c(0,0,0), c(0,1,0) - c(0,0,0), c(1,1,1) - c(0,0,0)), intercept=FALSE)
pred.r

forest(pred.r$pred, ci.lb=pred.r$ci.lb, ci.ub=pred.r$ci.ub, slab=c("Diff Temporal grouping","Diff Spatial grouping", "Diff Both"))

Now the CIs are quite narrow and exlude 0.

This aside, I would recommend that you include random effects for species twice, once without and once with the phylogenetic correlation matrix:

h$speciesIDnon <- h$speciesID
res <- rma.mv(zf, vzf, mods=~sce_temporal*sce_spatial, random = list (~1|effectsizeID, ~1|studyID, ~1|speciesIDnon, ~1|speciesID), R=list(speciesID=corr), data=h)

This is model (15) from:

Nakagawa, S., & Santos, E. S. A. (2012). Methodological issues and advances in biological meta-analysis. Evolutionary Ecology, 26(5), 1253-1274.

Conclusions do not change as far as I can tell, but I would still go with that model. A LRT also shows that this model fits significantly better:

res0 <- rma.mv(zf, vzf, mods=~sce_temporal*sce_spatial, random = list (~1|effectsizeID, ~1|studyID, ~1|speciesID), R=list(speciesID=corr), data=h)
anova(res, res0)

Best,
Wolfgang

_______________________________________________________

Prof. Dr. Rafael Rios Moura
scientia amabilis
Coordenador de Pesquisa e do NEPEE/CNPq
UEMG - Unidade Ituiutaba

ORCID:?http://orcid.org/0000-0002-7911-4734
Curr?culo Lattes: http://lattes.cnpq.br/4264357546465157
Research Gate: https://www.researchgate.net/profile/Rafael_Rios_Moura2
Rios de Ci?ncia:?https://www.youtube.com/channel/UCu2186wIJKji22ai8tvlUfg

---------- Forwarded message ---------
De: Rafael Rios <biorafaelrm at gmail.com>
Date: seg., 1 de jun. de 2020 ?s 16:53
Subject: Overlapping CIs with significant difference among subgroups
To: <r-sig-meta-analysis at r-project.org>, Viechtbauer Wolfgang (SP)
<wolfgang.viechtbauer at maastrichtuniversity.nl>

Dear Wolfgang and All,

I conducted a multilevel mixed-effects meta-analysis and found differences
between levels of two moderators. I was expecting to find non-overlapped
confidence intervals. However, I obtained overlapped confidence intervals
for all subgroups. How can I interpret these results? In such situation,
should I trust in the Q-test or in the CIs? I controlled for phylogenetic
non-independence. Is there a chance of this approach affect the estimation
of CIs using predict function? My dataset and script are attached.

Best wishes,
_______________________________________________________

Prof. Dr. Rafael Rios Moura
Coordenador de Pesquisa e do NEPEE/CNPq
Laborat?rio de Ecologia e Zoologia (LEZ)
UEMG - Unidade Ituiutaba

ORCID:?http://orcid.org/0000-0002-7911-4734
Curr?culo Lattes: http://lattes.cnpq.br/4264357546465157
Research Gate: https://www.researchgate.net/profile/Rafael_Rios_Moura2
Rios de Ci?ncia:?https://www.youtube.com/channel/UCu2186wIJKji22ai8tvlUfg

[R-meta] Overlapping CIs with significant difference among subgroups

Thread (3 messages)