[R-meta] Metareg: Significant reduction in tau-squared but no change in i-squared

Dear Richard,

See comments below.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:

r-sig-meta-analysis-bounces at r-project.org]

On Behalf Of Richard Simonson
Sent: Wednesday, 02 September, 2020 18:48
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] Metareg: Significant reduction in tau-squared but no
change in i-squared

I've conducted a meta-regression with metareg on a random-effects
meta-analysis and am attempting to describe the results. The output of the
model is at the end of my description

Based on the regression model, I'm given that our R-squared is 12%, which

I

take that to mean ~23% of the heterogeneity was accounted for when
covarying the model.

Not sure where you get the 23% from. The R^2 of 12% means that (an
estimated) 12% of the heterogeneity was accounted for (by the moderator(s)
included in the model).

I also see that there was a reduction in tau-squared from the random
effects meta-analysis to the meta-regression, but no change in I-squared.

Can someone help me understand why, with a positive non-zero r-squared,
there's a change in tau-squared but not in I-squared?

First of all, all these things (I^2 in a model without moderators, I^2 in
a meta-regression model, and R^2) are estimates, so they can simply be
'off'. For example, it can happen that tau^2 or I^2 actually increase when
including a moderator in a model, although in theory that doesn't make
sense.

Also, I^2 in a meta-regression model has a different interpretation than
in a model without moderators. In a model without moderators, I^2 is an
estimate how much of the total variability (which is composed of
heterogeneity and sampling variability) is due to heterogeneity. However,
in a meta-regression model, I^2 is an estimate how much of the *unaccounted
for variability* (which is composed of residual / unaccounted for
heterogeneity and sampling variability) is due to the residual /
unaccounted for heterogeneity. That's different than asking how much of the
*total variability* is due to residual / unaccounted for heterogeneity. So
I^2 in a meta-regression model is a bit of a strange statistic anyway and I
am not sure how many people actually grasp its correct interpretation.

As a more technical point: The way I^2 is computed in meta-regression
models is also a bit strange. The way the 'typical' sampling variance is
computed under such models actually depends on the moderators included in
the model, which one could argue is a bit odd. But given that people
typically compute I^2 with (Q-df)/df (where Q could be the test for
heterogeneity or the test for residual heterogeneity) this is what happens.

Which one is more feasible to report that we were able to account for some
of the heterogeneity?

In meta-regression models, I would report R^2 and not report I^2. You can
also report the test for residual heterogeneity, so if somebody feels the
need to compute I^2 based on that, they always can.

Thanks!

Meta output:
I-squared: 87%
tau-squared: .27

Meta-reg output
tau^2 (estimated amount of residual heterogeneity):     0.2426 (SE =

0.2570)

tau (square root of estimated tau^2 value):                    0.4925
I^2 (residual heterogeneity / unaccounted variability):    86.68%
H^2 (unaccounted variability / sampling variability):       7.51
R^2 (amount of heterogeneity accounted for):               12.25%