[R-meta] Asking for continuous moderating effects

Fri, Sep 10, 2021 4:09 AM

Thanks. The problem stems from the random-effects model (which needs to be fit to compute R^2) not converging. We can see this if we actually fit the model:

res <- rma(rAB, vAB, data=sample_moderator)

We can get it to converge by increasing the number of iterations:

res <- rma(rAB, vAB, data=sample_moderator, control=list(maxit=1000))

And then we do the same in the meta-regression model:

res <- rma(rAB, vAB, mods = ~ M1, data=sample_moderator, control=list(maxit=1000))
res

And now you got the R^2 value.

This wasn't your question, and I don't know any of the details of this analysis, but since I can see the dataset now, I would have several concerns about this analysis.

First, it seems like there are really two studies, the first providing 16 correlations, the second providing one. I do not know if the N=34 for the first study always refers to the same 34 individuals, but if so, these correlations definitely cannot be treated as independent (which they are at this point). Even if there is no overlap of participants, it's a bit of an unusual data structure.

Second, the 16 correlations from the first study are really four times the same set of four correlations. That is also rather peculiar.

Finally, a bubble plot based on the meta-regression model (regplot(res)) shows that the entire relationship between 'M1' and 'rAB' hinges on that single correlation from the second study. plot(influence(res)) shows how influential that one data point is.

Again, I do not know any of the details, but these data look very strange, the conclusion (that there is a relationship between M1 and rAB) is not robust, and the analysis is probably ignoring important dependencies.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On
Behalf Of Kim, Jaewoo
Sent: Friday, 10 September, 2021 0:18
To: r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Asking for continuous moderating effects

ATTACHMENT(S) REMOVED: sample_moderator.R | Moderator - sample data.xlsx

Thank you for your heads up, Professor Viechtbauer. I will keep that in mind. I
have attached the sample excel file and the sample code where the problem is to be
reproduced. Could you please take a look at these files? As I have a problem with
the result I below, I only included the sample data regarding result I. Please let
me know if you need anything else to reproduce. Many thanks.

I am analyzing continuous moderating effects with metafor. When I ran the
moderator analysis, I found that R^2 is NA%. Since my metafor is an older version,
I thought NA appeared in the results. Though I updated my metafor (v 3.0-2), the
results were still the same (please see the result I). However, when I ran another
moderator analysis with metafor, R^2 appeared 0, not NA (please see result II).
May I ask why this happens? In this case, can I view NA as 0%? Below are the
results with R code.

Results I

# Moderator 1
res <- rma(rAB, vAB, mods = ~ M1, data=sample_moderator)
res

Mixed-Effects Model (k = 17; tau^2 estimator: REML)

tau^2 (estimated amount of residual heterogeneity):???? 0.0069 (SE = 0.0112)
tau (square root of estimated tau^2 value):???????????? 0.0830
I^2 (residual heterogeneity / unaccounted variability): 22.42%
H^2 (unaccounted variability / sampling variability):?? 1.29
R^2 (amount of heterogeneity accounted for):??????????? NA%

Test for Residual Heterogeneity:
QE(df = 15) = 16.9198, p-val = 0.3237

Test of Moderators (coefficient 2):
QM(df = 1) = 9.4482, p-val = 0.0021

Model Results:

???????? estimate????? se???? zval??? pval??? ci.lb??? ci.ub
intrcpt?? -0.5843? 0.2530? -2.3094? 0.0209? -1.0802? -0.0884?? *
M1???????? 0.2672? 0.0869?? 3.0738? 0.0021?? 0.0968?? 0.4375? **

===================================================================

Results II

res <- rma(rAE, vAE, mods = ~ M1, data=sample_moderator)
res

Mixed-Effects Model (k = 10; tau^2 estimator: REML)

tau^2 (estimated amount of residual heterogeneity):???? 0.0291 (SE = 0.0194)
tau (square root of estimated tau^2 value):???????????? 0.1705
I^2 (residual heterogeneity / unaccounted variability): 77.99%
H^2 (unaccounted variability / sampling variability):?? 4.54
R^2 (amount of heterogeneity accounted for):??????????? 0.00%

Test for Residual Heterogeneity:
QE(df = 8) = 37.9082, p-val < .0001

Test of Moderators (coefficient 2):
QM(df = 1) = 0.9967, p-val = 0.3181

Model Results:

???????? estimate????? se???? zval??? pval??? ci.lb?? ci.ub
intrcpt??? 0.4235? 0.2087?? 2.0297? 0.0424?? 0.0146? 0.8325? *
M1??????? -0.0862? 0.0863? -0.9984? 0.3181? -0.2554? 0.0830

========================================================================

From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl>
Sent: Thursday, September 9, 2021 3:05 AM
To: Kim, Jaewoo <jkim at bauer.uh.edu>; r-sig-meta-analysis at r-project.org <r-sig-
meta-analysis at r-project.org>
Subject: RE: [R-meta] Asking for continuous moderating effects

Dear Jaewoo,

Please provide a reproducible example for the case where R^2 is reported as NA%.

Also, please switch of line wrapping in your email client (or whatever is causing
the results below to be so mangled up).

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On
Behalf Of Kim, Jaewoo
Sent: Thursday, 09 September, 2021 9:46
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] Asking for continuous moderating effects

Hello,

Hope that this email goes well. I am analyzing continuous moderating effects with
metafor. When I ran the moderator analysis, I found that R^2 is NA%. Since my
metafor is an older version, I thought NA appeared in the results. Though I
updated my metafor (v 3.0-2), the results were the same (please see the result

I).

However, when I ran another moderator analysis with metafor, R^2 appeared 0, not
NA (please see result II). May I ask why this happens? In this case, can I view

NA

as 0%? Below are the results with R code.

Results I

res <-
rma(rAB, vAB, mods = ~ M1, data=en_moderator)
res

Mixed-Effects
Model (k = 17; tau^2 estimator: REML)

tau^2
(estimated amount of residual heterogeneity):???? 0.0069 (SE = 0.0112)
tau (square
root of estimated tau^2 value):
0.0830
I^2 (residual
heterogeneity / unaccounted variability): 22.42%
H^2
(unaccounted variability / sampling variability):?? 1.29
R^2 (amount
of heterogeneity accounted for):
NA%

Test for
Residual Heterogeneity:
QE(df = 15) =
16.9198, p-val = 0.3237

Test of
Moderators (coefficient 2):
QM(df = 1) =
9.4482, p-val = 0.0021

Model
Results:

???????? estimate????? se
zval??? pval??? ci.lb
ci.ub
intrcpt?? -0.5843
0.2530? -2.3094? 0.0209
-1.0802? -0.0884?? *
M1???????? 0.2672
0.0869?? 3.0738? 0.0021
0.0968?? 0.4375? **

===================================================================

Results II

res <-
rma(rAE, vAE, mods = ~ M1, data=en_moderator)
res

Mixed-Effects
Model (k = 10; tau^2 estimator: REML)

tau^2
(estimated amount of residual heterogeneity):???? 0.0291 (SE = 0.0194)
tau (square
root of estimated tau^2 value):
0.1705
I^2 (residual
heterogeneity / unaccounted variability): 77.99%
H^2
(unaccounted variability / sampling variability):?? 4.54
R^2 (amount
of heterogeneity accounted for):
0.00%

Test for
Residual Heterogeneity:
QE(df = 8) =
37.9082, p-val < .0001

Test of
Moderators (coefficient 2):
QM(df = 1) =
0.9967, p-val = 0.3181

Model
Results:

???????? estimate????? se
zval??? pval??? ci.lb
ci.ub
intrcpt??? 0.4235
0.2087?? 2.0297? 0.0424
0.0146? 0.8325? *
M1??????? -0.0862
0.0863? -0.9984? 0.3181
-0.2554? 0.0830

[R-meta] Asking for continuous moderating effects

Thread (5 messages)