Message-ID: <fffea813bc6f4bf4a85105604ce1c1a3@UM-MAIL3214.unimaas.nl>
Date: 2020-06-02T06:48:07Z
From: Wolfgang Viechtbauer
Subject: [R-meta] Overlapping CIs with significant difference among subgroups
In-Reply-To: <CAAnf+jTVJ8q=-nN-Fdcn9=JB=50DCEutvUZGxo5u3wrhi2wqBg@mail.gmail.com>
Dear Rafael,
CIs can overlap and yet the difference between the two levels can be significant. See, for example:
https://towardsdatascience.com/why-overlapping-confidence-intervals-mean-nothing-about-statistical-significance-48360559900a?gi=b673a691634d
https://www.psychologicalscience.org/observer/understanding-confidence-intervals-cis-and-effect-size-estimation
https://blog.minitab.com/blog/real-world-quality-improvement/common-statistical-mistakes-you-should-avoid
and many more (just google for "test difference overlapping confidence intervals" or something along those lines). They don't talk about meta-analysis per se, but it's the same principle.
So, you can trust the test of the difference between the levels of the moderators.
Best,
Wolfgang
>-----Original Message-----
>From: Rafael Rios [mailto:biorafaelrm at gmail.com]
>Sent: Monday, 01 June, 2020 21:54
>To: r-sig-meta-analysis at r-project.org; Viechtbauer, Wolfgang (SP)
>Subject: Overlapping CIs with significant difference among subgroups
>
>ATTACHMENT(S) REMOVED: dataset.csv | pruned_super-tree.tre | script.R
>
>Dear Wolfgang and All,
>
>I conducted a multilevel mixed-effects meta-analysis and found differences
>between levels of two moderators. I was expecting to find non-overlapped
>confidence intervals. However, I obtained overlapped confidence intervals
>for all subgroups. How can I interpret these results? In such situation,
>should I trust in the Q-test or in the CIs? I controlled for phylogenetic
>non-independence. Is there a chance of this approach affect the estimation
>of CIs using predict function? My dataset and script are attached.
>
>Best wishes,
>_______________________________________________________
>
>Prof. Dr. Rafael Rios Moura
>Coordenador de Pesquisa e do NEPEE/CNPq
>Laborat?rio de Ecologia e Zoologia (LEZ)
>UEMG - Unidade Ituiutaba
>
>ORCID:?http://orcid.org/0000-0002-7911-4734
>Curr?culo Lattes: http://lattes.cnpq.br/4264357546465157
>Research Gate: https://www.researchgate.net/profile/Rafael_Rios_Moura2
>Rios de Ci?ncia:?https://www.youtube.com/channel/UCu2186wIJKji22ai8tvlUfg