[R-meta] sd of blups vs tau in RE model

Fri, Jun 30, 2023 3:20 AM

Please see below for my responses.

Best,
Wolfgang

But this is not true as demonstrated with the code I provided.

Please try out what happens when you change k to different values. It has no systematic effect on mean(blups$se^2).

I am a bit confused by your question, as you are not using the BLUPs to compute this; you are using the estimated coefficient and value of tau^2. Also, looking at the proportion larger than 0 isn't the best example because this will be ~50% eithre way. So let's compare:

pnorm(0.1, coef(res), sqrt(res$tau2), lower.tail=FALSE)
mean(blups$pred > 0.1)

As you will see, these are not the same. The latter yields a smaller proportion since the distribution of the BLUPs is not as wide as the estimated distribution of true effects based on the model estimates. We can seee this with:

hist(blups$pred, freq=FALSE, breaks=50)
curve(dnorm(x, coef(res), sqrt(res$tau2)), add=TRUE, lwd=3)

________________________________________
From: Viechtbauer, Wolfgang (NP) <wolfgang.viechtbauer at maastrichtuniversity.nl>
Sent: Friday, 30 June 2023 17:38
To: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis at r-
project.org>; James Pustejovsky <jepusto at gmail.com>
Cc: Yefeng Yang <yefeng.yang1 at unsw.edu.au>
Subject: RE: [R-meta] sd of blups vs tau in RE model

Dear Yefeng,

This was actually recently discussed on this mailing list:

https://stat.ethz.ch/pipermail/r-sig-meta-analysis/2023-May/004627.html

It is NOT true that the variance of the BLUPs will be equal to or approximate
tau^2 when k is large. As I explain in that post, one can decompose tau^2 into
two parts by the law of total variance. The variance of the BLUPs is only one
part of this. To demonstrate:

library(metafor)

tau2 <- .02
k <- 2500
vi <- runif(k, .002, .05)
yi <- rnorm(k, 0, sqrt(vi + tau2))

res <- rma(yi, vi)
res$tau2
blups <- ranef(res)

# variance of the BLUPs (way too small as this is essentially only
var(E(u_i|y_i)))
var(blups$pred)

# by adding what is essentially E(var(u_i|y_i)), we get (approximately) tau^2
var(blups$pred) + mean(blups$se^2)

The larger tau^2 is relative to the sampling variances, the less relevant
mean(blups$se^2) will be (try running the code above with tau2 <- .2). But still,
the variance of the BLUPs will underestimate tau^2.

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On
Behalf Of Yefeng Yang via R-sig-meta-analysis
Sent: Friday, 30 June, 2023 6:57
To: James Pustejovsky
Cc: Yefeng Yang; R Special Interest Group for Meta-Analysis
Subject: Re: [R-meta] sd of blups vs tau in RE model

Exactly. I was meant to the number of studies. I have no idea why I typed
within-
study replicates. Sorry for the confusion.
________________________________
From: James Pustejovsky <jepusto at gmail.com>
Sent: Friday, 30 June 2023 14:41
To: Yefeng Yang <yefeng.yang1 at unsw.edu.au>
Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis at r-
project.org>
Subject: Re: [R-meta] sd of blups vs tau in RE model

The approximations there are predicated on k (the number of studies) being large
enough that the estimated heterogeneity (tau-hat) converges to the true
heterogeneity parameter.

On Thu, Jun 29, 2023 at 11:29?PM Yefeng Yang
<yefeng.yang1 at unsw.edu.au<mailto:yefeng.yang1 at unsw.edu.au>> wrote:
Hi both,

I happen to come across a paper, which can answer both of your comments.

Eq. 1 and the following Eqs. show the derivation of the equivalence mentioned by
my earlier email.

Wang C C, Lee W C. A simple method to estimate prediction intervals and
predictive distributions: summarizing meta?analyses beyond means and confidence
intervals[J]. Research Synthesis Methods, 2019, 10(2): 255-266.

Best,
Yefeng
________________________________
From: James Pustejovsky <jepusto at gmail.com<mailto:jepusto at gmail.com>>
Sent: Friday, 30 June 2023 13:08
To: Yefeng Yang <yefeng.yang1 at unsw.edu.au<mailto:yefeng.yang1 at unsw.edu.au>>
Cc: R Special Interest Group for Meta-Analysis <r-sig-meta-analysis at r-
project.org<mailto:r-sig-meta-analysis at r-project.org>>
Subject: Re: [R-meta] sd of blups vs tau in RE model

Thanks for your clarification. Your explanations are very clear. Actually, the
SD of BLUPs and tau will converge when the within-study replicates are getting
large.

Can you say more about this? Is this claim based on simulations or something? I
see the intuition, but it also seems like this property might depend not only on
the within-study replicates all being large, but also on their _relative_ sizes.

[R-meta] sd of blups vs tau in RE model

Thread (15 messages)