Skip to content

[R-meta] Problems using rma.mh

9 messages · David Fisher, Gerta Ruecker, Wolfgang Viechtbauer

David Fisher
# 
Dear metafor community,

I am a Stata user with only limited knowledge of R.  I am trying to
validate some Stata code by comparing with output from metafor; in
particular, sparse data/zero cells using Mantel-Haenszel methods.

My reference is Wolfgang's page
"http://www.metafor-project.org/doku.php/tips:comp_mh_different_software".
If I type:

library(metafor)
dat <- dat.nielweise2007
rma.mh(measure="OR", ai=ai, n1i=n1i, ci=ci, n2i=n2i, data=dat)

...then I successfully obtain the output shown on the webpage.
However, if I alter the command in (seemingly) any way, I get the
following error:

Error in sprintf(format, names(x)) :
  invalid format '%NA'; use format %s for character objects

For instance, if I limit the dataset to studies with zero events in
the treatment arm:

rma.mh(measure="OR", ai=ai, n1i=n1i, ci=ci, n2i=n2i,
data=dat2007[which(dat2007$ai==0),])

...and similarly if I attempt to use another data frame, or import
from Stata using "read.dta".  Furthermore, I do not appear to have the
same issue if I use another model.  For instance, the following runs
with no problem:

rma(measure="OR", ai=ai, n1i=n1i, ci=ci, n2i=n2i,
data=dat2007[which(dat2007$ai==0),])

I am using a fresh install of R 4.0.1 and of the metafor package.

Could someone point me in the right direction?

Many thanks,

David.
Gerta Ruecker
# 
Dear David,

Your code defines a data set called dat, but not dat2007. But this 
doesn't seem to be the problem (I can't locate). Maybe the double zero 
is the problem?

You could try the meta package:

library(metafor)
dat2007 <- dat.nielweise2007
library(meta)
m1 <- metabin(ai, n1i, ci, n2i, data = dat2007)
m2 <- metabin(ai, n1i, ci, n2i, data=dat2007[dat2007$ai==0,])
m1
m2

Best,

Gerta


Am 12.06.2020 um 11:01 schrieb David Fisher:

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Gerta Ruecker
    

    

      
      #
    

  


  

      
Once more. Here is the code with OR and without continuity correction:

library(metafor)
dat2007 <- dat.nielweise2007
library(meta)
m1 <- metabin(ai, n1i, ci, n2i, sm = "OR", incr = 0, data = dat2007)
m2 <- metabin(ai, n1i, ci, n2i, sm = "OR", incr = 0, 
data=dat2007[dat2007$ai==0,])
m1
m2

For m2, with only zero events, you obtain OR = 0, but without estimates 
for the standard error and thus without confidence intervals. See

forest(m2)

Best,

Gerta

Am 12.06.2020 um 11:15 schrieb Gerta Ruecker:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      David Fisher
    

    

      
      #
    

  


  

      
Dear Gerta,
Many thanks for your reply.

Firstly: apologies, I was inconsistent in the naming of my dataset in
my example code, as you spotted.  But no, this doesn't solve my
original problem :-)
When you say "I cannot locate", do you mean you do not see the same
error message?

Thankyou for pointing me towards the alternative package "metabin".
Can you tell me whether it can be used to calculate the Breslow-Day
(and/or Tarone) statistics for homogeneity of odds ratios?   Or, could
you tell me whether you would expect these statistics to be calculated
using raw counts, or using continuity-corrected counts?

Thanks again,
David.



On Fri, Jun 12, 2020 at 10:24 AM Gerta Ruecker

        
<ruecker at imbi.uni-freiburg.de> wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Gerta Ruecker
    

    

      
      #
    

  


  

      
Dear David,

With "I can't locate" I meant that I see the same error, but don't know 
why :-(

With respect to the homogeneity statistics you asked for: I don't think 
they are implemented, this would be a question for Guido (who will be 
back at work on Monday). I don't find any hint to them when typing 
help(metabin).

In principle, I think it preferable to avoid continuity corrections.

Maybe Wolfgang can solve the problem with rma.mh and also answer the 
other question?

Best,

Gerta

Am 12.06.2020 um 11:45 schrieb David Fisher:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      David Fisher
    

    

      
      #
    

  


  

      
Dear Gerta,

Thanks -- it's good at least to know that it's not only me seeing the error :-)

Regarding continuity correction: I agree;  but given that a correction
is sometimes applied even to Mantel-Haenszel pooling in extreme cases
(such as when all studies have zero events in the same arm, as in my
example), I wondered whether a similar argument could/should be
applied to other associated statistics.

I have copied Wolfgang into this email:  Wolfgang, could you take a
look at the issue with rma.mh, please?

Many thanks,

David.



On Fri, Jun 12, 2020 at 10:54 AM Gerta Ruecker

        
<ruecker at imbi.uni-freiburg.de> wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Wolfgang Viechtbauer
    

    

      
      #
    

  


  

      
Hi David, Hi Gerta,

Thanks for the note, David. This was an oversight in the printing function. When there are no events at all in one of the arms across all studies, then certain statistics cannot be computed, which are then NA. This leads to problems when trying to format these values with sprintf(). I'll fix this asap.

But rma.mh() itself works, so this is fine:

dat <- dat.nielweise2007
res <- rma.mh(measure="OR", ai=ai, n1i=n1i, ci=ci, n2i=n2i, data=dat[which(dat$ai==0),])

If you are interested in the Breslow-Day and Tarone statistics, you can still get them from the object with:

res$BD
res$TA

and the corresponding p-values with:

res$BDp
res$TAp

But in this case, you will find that both statistics are actually NA. In other words, these tests cannot be conducted when there are no events in one of the two arms. Or one would have to use continuity corrections, but the point of the MH method and related statistics (such as BD and TA) is that they do not require such corrections and in fact were never meant to be combined with such corrections.

Interestingly, the Cochran-Mantel-Haenszel test can still be conducted:

res$CO
res$COp

Best,
Wolfgang

            

      
      
      
    

  
  


  


      

    

    
3 days later

    

      
      

        


  

        

      David Fisher
    

    

      
      #
    

  


  

      
Hi Wolfgang,

Aha!  It didn't occur to me that the error might be referring to the
output, rather than to the input.  That makes total sense...and
thankyou for the suggestion to obtain the results directly from the
object!

I have had a play with subsets of studies and with the add(), to() and
drop00 options, and it seems that the following is true:
  a) If the M-H pooling is done using corrected counts [ via e.g.
add=c(0, 0.5) to=c("only0", "only0") ] then the Breslow-Day (+/-
Tarone) statistic is also calculated using corrected counts, and hence
has a defined value;
  b) If effect sizes for the individual studies are *not* calculated
using corrected counts [ again, via add=c(0, 0.5) to=c("only0",
"only0") ]  then the Q statistic is zero/undefined.

Finally, regarding your observation that the Cochran-Mantel-Haenszel
test can still be conducted, even when all studies have zero events in
the same arm:  this is presumably because the CMH test is very similar
to a Peto chi-squared test, which can be fitted in this scenario, e.g.

res.peto <- rma.peto(ai=ai, n1i=n1i, ci=ci, n2i=n2i,
data=dat[which(dat$ai==0),])

Many thanks for your work developing "metafor", and for your time
answering my questions.

Best wishes,

David.


On Fri, Jun 12, 2020 at 11:58 AM Viechtbauer, Wolfgang (SP)

        
<wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Wolfgang Viechtbauer
    

    

      
      #
    

  


  

      
I've fixed the printing issue in the devel version:

https://github.com/wviechtb/metafor#installation

With respect to your notes:

a) Sounds right. Whether the resulting tests work properly then is another question.

b) Correct. Without some adjustment, the individual log odds ratios cannot be computed and neither can the Q-statistic then.

Best,
Wolfgang