Skip to content

[R-meta] Reproducing results using regtest in metafor

3 messages · James Pustejovsky, Sutton, Alex (Prof.)

Sutton, Alex (Prof.)
# 
Dear James

Thank you for taking the time to think about this and write. I think your approach of writing the model out is a good one for discussing exactly what is going on "under the hood".

Your justification of the degrees of freedom seems sound - much appreciated.

I am still a little uncertain about the model specification because an additive random effect is specified in the initial meta-regression using the rma command. But then a multiplicative error is specified in the regtest command. These probably dont have any bearing on the degrees of freedom, but I would greatly appreciate it if someone could explain how these are jointly implemented.

With very best wishes

Alex
2 days later
James Pustejovsky
# 
Alex,

Looking at the source code for regtest.rma, it looks like my previous
syntax can be simplified even further, to simply:
lm_fit <- lm(yi ~ m.c + sqrt(vi), data = ma.dataset, weights = 1 / vi)
summary(lm_fit)

The random effects structure / weighting scheme used to fit the initial
metaregression is simply disregarded when re-fitting the model. I think
that's reasonable because it forces one to make a coherent assumption about
the structure of the errors in the regression. In contrast, the other
approach involves fitting the first model based on random effects
weighting, and then fitting the Egger regression on the residuals, but
using a *different* weighting scheme. I can't think of any data generating
process where this would be an efficient way to fit the model or a valid
approach for inference. Thus, it would seem preferable to fit the model all
at once, using assumptions that can at least be clearly stated.

James

On Mon, Aug 12, 2019 at 7:34 AM Sutton, Alex (Prof.) <ajs22 at leicester.ac.uk>
wrote:

  
  


  


      

    

    

      
      

        


  

        

      Sutton, Alex (Prof.)
    

    

      
      #
    

  


  

      
Thanks James

Again, your response is super helpful.

(I did have a dig into the code of regtest but wasn't confident I knew enough about it to reliably unpick it). The point you make about coherence is important. It is interesting to note that the multiplicative error of the test as specified (originally by Egger) was due to it being conceived with respect to a Galbraith plot (which has standardised effect on one axis so regression is "divided through" by the standard error) not any real belief about the error structure - but it has stuck and does deal with the scenario of underdispersion (less variability than expected by chance even under fixed effect) which could occur if the most extreme study results under a fixed effect assumption have been suppressed.

So while the test as regtest conducts it does not mix errors, there is still the issue that the analysis strategy does (i.e. why use different error structures for baseline risk on its own and when adding in the standard error for asymmetry testing?). One thing in favour of the "2-stage" analysis is that it allows the construction of a funnel based on the residuals to see how its appearance changes as a result of adjusting for the initial covariate.

Thanks again for all your help - I now understand what is going on and thus you have comprehensively answered my question.

Very best wishes

Alex