Skip to content

[R-meta] Difference between subset (in a loop) and mods in metafor rma.mv

6 messages · Wolfgang Viechtbauer, Reza Norouzian, Brendan Pearl

#
I am trying to understand the difference using the 'subset' and 'mods' options in rma.mv.

I have run two analyses using rma.mv on the same dataset that has a three level structure (articles nested within larger studies) and get different results.

In the first example I have given below, is looping over rma.mv and passing a different predictor to the subset option a univariate meta-analysis? And in the second example, is passing the predictor to the mods option running a multivariable meta-analysis?

Study <- c("A","A","B","C","C","D","E","F","F","G")
Article <- c("1","2","3","4","5","6","7","8","9","10")
Predictor <- c("x","x","x","y","y","y","x","x","y","y")
yi <- c(-.2,-.3,-.8,.5,.6,.4,-.1,-.8,.3,.8)
vi <- c(.01,.01,.01,.01,.01,.01,.01,.01,.01,.01)

dat <- data.frame(Study,Article,Predictor,yi,vi)

#FIRST EXAMPLE#Subset analyses
# gives me

#1   x OR = -0.49 [-0.84, -0.13]
#2   y    OR = 0.51 [0.31, 0.72]

df_subset <- data.frame(val=as.character(),subset_result=as.character())

for (val in unique(dat$Predictor)){

res_univariate <- rma.mv(yi=yi,
V=vi,
data=dat,
random = ~1 | Study/Article,
subset = Predictor == val,
method = "REML"
)

result <- capture.output(cat(
"OR = ",
format(round(res_univariate$beta, 2),nsmall = 2),
" [",
format(round(res_univariate$ci.lb, 2),nsmall = 2),
", ",
format(round(res_univariate$ci.ub, 2),nsmall = 2),
"]",
sep=""
)
)

df_subset[nrow(df_subset) +1,] = c(
val,
result
)

}

#SECOND EXAMPLE
#Mod analysis
#gives me:

#                      estimate   ci.lb    ci.ub

#factor(Predictor)x   -0.4825    -0.7246  -0.2404
#factor(Predictor)y    0.5807     0.3386   0.8229

df_mod <- data.frame(val=as.character(),mod_result=as.character())

res_multivariate <- rma.mv(yi=yi,
V=vi,
data=dat,
random = ~1 | Study/Article,
mods = ~ factor(Predictor)-1,
method = "REML"
)
summary(res_multivariate)
Message-ID: <fIfSRRMzFrWRvOdZVkqSukE2VgluGEjoXbdErjGGvI0d_ov_LBuvb5UxW9d3EzQrqY5w56On_-HG1VIMltaPHFmh9Idxn1fdZz42m0NQwGE=@brendans-bits.com>
#
Dear Brendan,

Both approaches fit multilevel models. The difference arises because of two reasons:

1) When using the full dataset and using 'Predictor' as a moderator, then it is possible for information to be shared/borrowed from both levels of the moderator within studies. 

2) When subsetting based on 'Predictor', you allow the variance components to differ across the two levels of the moderator. This is essentially what is discussed here:

https://www.metafor-project.org/doku.php/tips:comp_two_independent_estimates

but in the context of a simpler model.

Best,
Wolfgang
#
Thankyou Wolfgang,

Just for my understanding: "When using the full dataset and using 'Predictor' as a moderator, then it is possible for information to be shared/borrowed from both levels of the moderator within studies." - is this what occurs in a multivariable/multivariate meta-analysis?

Regards,
Brendan.

------- Original Message -------
On Friday, September 29th, 2023 at 8:56 PM, Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

            
Message-ID: <tc8X6hrFLoNOy1LEPHIXrMrEGMevf5P9OGfemrjCqxyCSwJKatZPVnMPKbDuNVWTcLoYtVXQ9fwOSbDJPUeTKK6VAqC7Oeo2JtePLPl26nE=@brendans-bits.com>
#
Yes. For some relevant literature, see for example:

Riley, R. D., Abrams, K. R., Lambert, P. C., Sutton, A. J., & Thompson, J. R. (2007). An evaluation of bivariate random-effects meta-analysis for the joint synthesis of two correlated outcomes. Statistics in Medicine, 26(1), 78-97. https://doi.org/10.1002/sim.2524

Jackson, D., White, I. R., Price, M., Copas, J., & Riley, R. D. (2017). Borrowing of strength and study weights in multivariate and network meta-analysis. Statistical Methods in Medical Research, 26(6), 2853-2868. https://doi.org/10.1177/0962280215611702

Copas, J. B., Jackson, D., White, I. R., & Riley, R. D. (2018). The role of secondary outcomes in multivariate meta-analysis. Journal of the Royal Statistical Society, Series C, 67(5), 1177-1205. https://doi.org/10.1111/rssc.12274

Best,
Wolfgang
#
Also, Brenden, if you want to replicate the results from when you subset
the data (i.e., not allowing the borrowing of info.) with a single model,
then you can do:

rma.mv(yi~Predictor+0, vi, random = list(~Predictor | Study, ~Predictor |
Article),
       struct = c("DIAG","DIAG"), data=dat)

which for your toy dataset, where predictor's categories have no
heterogeneity at the Article level, may be reduced to:

rma.mv(yi~Predictor+0, vi, random = ~Predictor | Study,
       struct = "DIAG", data=dat)

This topic has come up on the list a number of times, you may want to scan
through the listserv's archives for related examples (e.g.,
https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2022-June/004074.html).

Reza


On Fri, Sep 29, 2023 at 11:25?AM Viechtbauer, Wolfgang (NP) via
R-sig-meta-analysis <r-sig-meta-analysis at r-project.org> wrote:

            

  
  
#
Thankyou both.

------- Original Message -------
On Saturday, September 30th, 2023 at 3:04 AM, Reza Norouzian via R-sig-meta-analysis <r-sig-meta-analysis at r-project.org> wrote:

            
Message-ID: <tgaty4O6YhQwy62Vwe6KLS2lydnEIqu0N5m-m2JJ7SbsWf8GlQKrLv6gcenhzzriufnWK-HHl8wfhwVDie9Ya_DOe3cRgvWJ2iRAB5WSsBY=@brendans-bits.com>