[R-meta] Extracting pooled rma.mv model summary statistics after multiple imputation

-----Original Message-----
From: Thomas Swanton [mailto:thomas.swanton at sydney.edu.au]
Sent: Friday, 10 December, 2021 2:51
To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis at r-project.org
Subject: Re: Extracting pooled rma.mv model summary statistics after multiple
imputation

Hi Wolfgang,

Thanks again for your helpful response.

If I may, I have another question - how can I calculate the pooled variance
inflation factors for each moderator to test for collinearity (again, after using
mice for multiple imputation)?

Many thanks,
Tom


From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl>
Sent: Sunday, 14 November 2021 1:58 AM
To: Thomas Swanton <thomas.swanton at sydney.edu.au>; r-sig-meta-analysis at r-
project.org <r-sig-meta-analysis at r-project.org>
Subject: RE: Extracting pooled rma.mv model summary statistics after multiple
imputation

Hi Thomas,

Thanks for the example. This leads to an error though in this step:

pool <- summary(pool(fit))

Error in h(simpleError(msg, call))
:

? error in evaluating the argument 'object' in selecting a method for function
'summary': the condition has length > 1

To be precise, by default, you will just get a bunch of warnings, but I use:

Sys.setenv("_R_CHECK_LENGTH_1_CONDITION_"="true")

which ends up switching the warnings here to errors.

The problem really comes from:

pool(fit)

Error in if (x$ddf > 0) qt(level/2, df = x$ddf, lower.tail = FALSE) else NA
:
? the condition has length > 1

And this error comes from a bug in 'broom' which is used by 'mice'. The problem
is in tidy.rma(), namely this line:

https://protect-au.mimecast.com/s/GOuDCjZ1N7inN41D0uW2cOp?domain=github.com

crit <- if (x$ddf > 0) qt(level/2, df=x$ddf, lower.tail=FALSE) else NA

In rma.mv() models with test="t", x$ddf is a vector, not a single value, and
hence we get an if() conditions that has length > 1. I will report this to the
broom authors.

As for your actual question, see this:

https://protect-au.mimecast.com/s/Py6MCk81N9tOGy3AvsVz45E?domain=stat.ethz.ch

So you could do:

round(apply(sapply(fit$analyses, \(x) x$sigma2), 1, mean), 4)
format.pval(median(sapply(fit$analyses, \(x) x$QEp)))
format.pval(median(sapply(fit$analyses, \(x) x$QMp)))

Best,
Wolfgang

-----Original Message-----
From: Thomas Swanton [mailto:thomas.swanton at sydney.edu.au]
Sent: Thursday, 11 November, 2021 1:18
To: Viechtbauer, Wolfgang (SP); r-sig-meta-analysis at r-project.org
Subject: Re: Extracting pooled rma.mv model summary statistics after multiple
imputation

Dear Wolfgang,

Thank you for your reply, and my apologies for the delayed response - I was on
leave for a few weeks.

Below I provide a minimal and fully reproducible example using the Bangert-

Drowns

et al. sample data following your blog post at https://www.metafor-
project.org/doku.php/tips:multiple_imputation_with_mice_and_metafor

I am conducting a three-level mixed-effects meta-regression with multiple effect
sizes (level 2; defined in the below example by "id") nested within studies
(level 3; defined in the below example by "author").

Where I am stuck is: how do I get the portion of the output about variance
components, heterogeneity, and the test for moderators after using multiple
imputation? The pooling gives me the model results (estimate, std error, t
statistic, df, p-value), but how do I pool the portion of output that is

normally

above "Model Results"?

In the last line of the code (fit$analyses[1]), I can access this for each
imputed dataset, but I am unsure how to access the pooled summary statistics.

I hope this clarifies where I am stuck. I would be very grateful for your

advice.

Many thanks,

Tom Swanton
PhD Candidate
School of Psychology, The University of Sydney

# load packages

library(metafor)
library(mice)

# use data from meta-analysis by Bangert-Drowns et al. (2004)

dat <- dat.bangertdrowns2004

# keep variables needed for analysis

dat <- dat[c("id", "author", "yi", "vi", "length", "wic", "feedback", "info",
"pers", "imag", "meta")]

# turn dummy variables into factors

dat$wic <- factor(dat$wic)
dat$feedback <- factor(dat$feedback)
dat$info? <- factor(dat$info)
dat$pers? <- factor(dat$pers)
dat$imag? <- factor(dat$imag)
dat$meta? <- factor(dat$meta)

# set up predictor matrix for imputations

predMatrix <- make.predictorMatrix(dat)
predMatrix[,"id"] <- 0 # don't use effect size id for imputing
predMatrix[,"author"] <- 0 # don't use author for imputing
predMatrix[,"vi"] <- 0 # don't use vi for imputing
predMatrix["id",] <- 0 # don't impute id (no NA's)
predMatrix["author",] <- 0 # don't impute author (no NA's)
predMatrix["yi",] <- 0 # don't impute yi (no NA's)
predMatrix["vi",] <- 0 # don't impute vi (no NA's)
predMatrix

# specify imputation method

impMethod <- make.method(dat)
impMethod

# generate multiple imputations

imp <- mice(dat, print=FALSE, m=20, predictorMatrix=predMatrix,

method=impMethod,

seed=1234)

# fit model of interest to each of the 20 imputed datasets

fit <- with(imp,
??????????? rma.mv(yi = yi,
?????????????????? V = vi,
?????????????????? slab = author,
?????????????????? random = ~ 1 | author/id,
?????????????????? test = "t",
?????????????????? method = "REML",
?????????????????? mods = ~ length + wic + feedback + info + pers + imag +

meta))

# pool and round results

pool <- summary(pool(fit))
pool[-1] <- round(pool[-1], digits=4)
pool

# view output from first imputed dataset

fit$analyses[1]

From: Viechtbauer, Wolfgang (SP) <wolfgang.viechtbauer at maastrichtuniversity.nl>
Sent: Saturday, 16 October 2021 12:18 AM
To: Thomas Swanton <thomas.swanton at sydney.edu.au>; r-sig-meta-analysis at r-
project.org <r-sig-meta-analysis at r-project.org>
Subject: RE: Extracting pooled rma.mv model summary statistics after multiple
imputation

Hi Tom,

Could you provide a minimal and fully reproducible example (see:

https://protect-

au.mimecast.com/s/-pXHCnx1jni7Z8PPPu9Oavy?domain=stat.ethz.ch) that can be used
to discuss this? Right now, it's not entirely clear to me where exactly you are
stuck.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On
Behalf Of Thomas Swanton
Sent: Friday, 15 October, 2021 2:02
To: r-sig-meta-analysis at r-project.org
Subject: [R-meta] Extracting pooled rma.mv model summary statistics after
multiple imputation

Dear colleagues,

Is anyone able to help me understand how to extract pooled rma.mv model summary
statistics after applying multiple imputation using mice please?

I am conducting a three-level mixed effects multiple meta-regression using the
rma.mv function in metafor, and have applied multiple imputation using mice
following the instructions available here: https://www.metafor-
project.org/doku.php/tips:multiple_imputation_with_mice_and_metafor

I have the table of pooled model coefficients as in the worked example, but I
haven't been able to work out how to extract pooled model summary statistics
(e.g., tau-squared, Q statistic).

Are you able to help me understand how to pool them across all the imputed
datasets please?

Many thanks,
Tom

Thomas Swanton, PhD Candidate
School of Psychology, The University of Sydney, Australia
thomas.swanton at sydney.edu.au

Tom Swanton?| PhD Candidate and Research
Assistant

The University of Sydney

Faculty of Science, School of Psychology, Brain & Mind Centre, Gambling

Treatment

& Research Clinic

Rm 204, Level 2, 94 Mallett St, Camperdown | NSW | 2050

+61 432 891 501

thomas.swanton at sydney.edu.au?|?sydney.edu.au

Twitter:?@TomSwanton?|
ResearchGate:?researchgate.net/profile/Thomas-Swanton

Editorial Assistant,?International Gambling Studies

We acknowledge the tradition of custodianship and law of the Country on which

the

University
of Sydney campuses stand. We pay our respects to those who have cared and
continue to care for Country.