Skip to content

Metafor -can't calculate heterogeneity with non-positive sampling variances

4 messages · Owen, Branwen, Viechtbauer Wolfgang (STAT), Michael Dewey

Owen, Branwen
# 
Hi, I'm doing a meta-analysis in metafor. All is fine except when there are 0s in the values that i'm pooling, then i get a pooled estimate but not the I2 that i am also interested in.
for example:

summary(rma.1<-rma(yi,vi,data=mix,method="ML",knha=F,weighted=F,intercept=T))
(where yi are the study outcomes, one of which is 0, and vi is the variance of the study outcomes)

Random-Effects Model (k = 17; tau^2 estimator: ML)

  logLik  deviance       AIC       BIC      AICc  
 13.0539       Inf  -22.1077  -20.4413  -21.2506  

tau^2 (estimated amount of total heterogeneity): 0.0119 (SE = 0.0043)
tau (square root of estimated tau^2 value):      0.1089

Model Results:

estimate       se     zval     pval    ci.lb    ci.ub          
  0.1837   0.0274   6.7154   <.0001   0.1301   0.2374      ***

---
Signif. codes:  0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1

Warning messages:
1: In rma(yi, vi, data = mix, method = "ML", knha = F, weighted = F,  :
  There are outcomes with non-positive sampling variances.
2: In rma(yi, vi, data = mix, method = "ML", knha = F, weighted = F,  :
  Cannot compute Q-test, I^2, or H^2 with non-positive sampling variances.

Is there any way around this?
thanks
Branwen
Viechtbauer Wolfgang (STAT)
# 
The warning message pretty much says it: When one of the variances is zero, then the I^2 statistic (and various other things) cannot be computed, at least if one sticks to the usual equations/methods. So, if you think the 0 sampling variances really make sense and you really want to get something like I^2, you will have to come up with a creative solution.

On the metafor package website, I explain how I^2 is computed (for the random-effects model):

http://www.metafor-project.org/doku.php/faq#how_are_i_2_and_h_2_computed_i

The crux of the problem is how to compute the 'typical' within-study variance (s^2). With any vi=0, you get division by zero in the equation given. So, you will have to compute s^2 in a different way. You could leave out the studies where vi=0, but this doesn't seem quite right, because this will inflate s^2. You could just take the simple average of the vi values and use that for s^2, but then it's not really I^2 anymore (it's I^2-like).

My question would be: How come you have studies where the sampling variance is estimated to be zero and does that really make sense? Maybe the solution is not to fix the computation of I^2, but to consider if vi=0 is really sensible.

Best,
Wolfgang
Owen, Branwen
# 
Thank you very much for your quick reply Wolfgang. The 0 does make sense - I'm working on some behavioural data and in one study none reported that particular behaviour. Up til now I have been calculating I2 without that study, as it's on the small side I don't think it makes much difference. I'm just beginning to wonder if there is another way. 

I'll think about it some more
best wishes
Branwen
Michael Dewey
#
At 16:38 27/08/2014, Owen, Branwen wrote:
Branwen
I have not tried this but if your dataset contains proportions as 
your text implies then why not supply xi and ni to rma.uni and then 
get it to compute one of the range of options outlined in the 
documentation for escalc? The Freeman-Tukey one might be worth considering.

Of course I may have misunderstood your brief description.
Michael Dewey
info at aghmed.fsnet.co.uk
http://www.aghmed.fsnet.co.uk/home.html