Skip to content

[R-meta] Funnel Plots for Multilevel Meta

9 messages · Dylan Johnson, Michael Dewey, Wolfgang Viechtbauer +2 more

Dylan Johnson
# 
Hello,

Thanks everyone for helping me sort out the Egger?s test with multi-level meta modelling!

Is there any option in R for producing Funnel plots that are appropriate for the nonindependence? I imagine that the standard funnel plot would be deceiving if it came from a multi-level design.

Many thanks!

Best,
Dylan


Dylan Johnson, MSc

MA Student, School and Clinical Child Psychology
Department of Applied Psychology and Human Development

University of Toronto
252 Bloor Street West

Toronto, ON M5S 1V6
Michael Dewey
# 
Dear Dylan

Perhaps I misunderstand you but if you have the data for a regression 
type test like Egger's do you not just plot that? The funnel() function 
in metafor does that and I am sure equivalent solutions can be found in 
meta and many other packages.

Michael
On 11/12/2020 01:14, Dylan Johnson wrote:

  
    

      
      
    

  


  


      

    

    

      
      

        


  

        

      Dylan Johnson
    

    

      
      #
    

  


  

      
Hi Michael,



Would it not be nonsensical to have multiple effects from the same article in the funnel plot though?



With the Egger?s regression I was able to accommodate the fact that their is nonindependence of the effects, but am unsure how to proceed with a funnel plot.



Dylan



Dylan Johnson, MSc

MA Student, School and Clinical Child Psychology
Department of Applied Psychology and Human Development

University of Toronto
252 Bloor Street West

Toronto, ON M5S 1V6



From: Michael Dewey<mailto:lists at dewey.myzen.co.uk>
Sent: December 11, 2020 6:12 AM
To: Dylan Johnson<mailto:dylanr.johnson at mail.utoronto.ca>; r-sig-meta-analysis at r-project.org<mailto:r-sig-meta-analysis at r-project.org>
Subject: Re: [R-meta] Funnel Plots for Multilevel Meta



EXTERNAL EMAIL:

Dear Dylan

Perhaps I misunderstand you but if you have the data for a regression
type test like Egger's do you not just plot that? The funnel() function
in metafor does that and I am sure equivalent solutions can be found in
meta and many other packages.

Michael

        
On 11/12/2020 01:14, Dylan Johnson wrote:

            

      
      
      
    

        

      
      
    

        
--
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

  
  


  


      

    

    

      
      

        


  

        

      Wolfgang Viechtbauer
    

    

      
      #
    

  


  

      
A funnel plot is simply a plot of the estimates against their standard errors (or some other measure of precision). So one can draw such a plot whether there are multiple estimates from the same study or not. Hence, funnel() in metafor will happily do so:

library(metafor)

dat <- dat.konstantopoulos2011
res <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat)
res
funnel(res)

One could indicate (with different colors or plotting symbols) which estimates belong to the same study. 

cols <- palette.colors(length(unique(dat$district)), palette="Alphabet")
cols <- cols[as.numeric(factor(dat$district))]
funnel(res, col=cols)

Then one can see how points from the same study (or in this case, 'district') cluster together.

To what extent such a plot is indicative of publication bias / small study effects is a different issue (but the same applies even to simpler meta-analyses with a single estimate per study).

Best,
Wolfgang

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Valeria Ivaniushina
    

    

      
      #
    

  


  

      
Dear Michael and Wolfgang.

at the end of Egger's regression / funnel plot thread, Wolfgang said: "To
what extent such a plot is indicative of publication bias / small study
effects is a different issue"

Could you please say more on this topic?
Particularly I am interested in this: is a plot with 8 effects informative?
Or a plot with less than 20 effects?

Best,
Valeria

On Fri, Dec 11, 2020 at 11:28 PM Viechtbauer, Wolfgang (SP) <

        
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

        

            

      
      
      
    

        

      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Gerta Ruecker
    

    

      
      #
    

  


  

      
Dear Valeria,

You find discussion of these issues in the literature, for example:

https://www.bmj.com/content/343/bmj.d4002 (Sterne et al. 2011, 
Recommendations for examining and interpreting funnel plot asymmetry in 
meta-analyses) - One rule of thumb is that looking at a funnel plot 
doesn't make much sense with less than 10 effects.

https://onlinelibrary.wiley.com/doi/full/10.1002/jrsm.1452 (Marks-Anglin 
& Chen 2020, A historical review of publication bias)

https://onlinelibrary.wiley.com/doi/10.1002/jrsm.1468 (Page et al. 2020, 
Investigating and dealing with publication bias and other reporting 
biases in meta?analyses of health research: A review)

Best,

Gerta

Am 12.12.2020 um 14:43 schrieb Valeria Ivaniushina:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Michael Dewey
    

    

      
      #
    

  


  

      
Dear Valeria

Whatever choice you make, and Gerta has given you some valuable 
references, make sure you actually look at the plot with an open mind 
and interpret everything in it. It is just another way of looking at the 
data and may show other features than small study effects.

Michael

        
On 12/12/2020 13:43, Valeria Ivaniushina wrote:

            

      
      
      
    

        

      
      
    

        

  
    
  


  


      

    

    

      
      

        


  

        

      Valeria Ivaniushina
    

    

      
      #
    

  


  

      
Dear Gerta,

Thank you so much for the references!

Best,
Valeria

On Sat, Dec 12, 2020 at 5:41 PM Dr. Gerta R?cker <

        
ruecker at imbi.uni-freiburg.de> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
  


  


      

    

    
3 days later

    

      
      

        


  

        

      Dylan Johnson
    

    

      
      #
    

  


  

      
Thank for the advice, I will try integrating the different colour into the clusters.



Best,

Dylan



Dylan Johnson, MSc

MA Student, School and Clinical Child Psychology
Department of Applied Psychology and Human Development

University of Toronto
252 Bloor Street West

Toronto, ON M5S 1V6



From: Viechtbauer, Wolfgang (SP)<mailto:wolfgang.viechtbauer at maastrichtuniversity.nl>
Sent: December 11, 2020 3:28 PM
To: Dylan Johnson<mailto:dylanr.johnson at mail.utoronto.ca>; Michael Dewey<mailto:lists at dewey.myzen.co.uk>; r-sig-meta-analysis at r-project.org<mailto:r-sig-meta-analysis at r-project.org>
Subject: RE: [R-meta] Funnel Plots for Multilevel Meta



EXTERNAL EMAIL:

A funnel plot is simply a plot of the estimates against their standard errors (or some other measure of precision). So one can draw such a plot whether there are multiple estimates from the same study or not. Hence, funnel() in metafor will happily do so:

library(metafor)

dat <- dat.konstantopoulos2011
res <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat)
res
funnel(res)

One could indicate (with different colors or plotting symbols) which estimates belong to the same study.

cols <- palette.colors(length(unique(dat$district)), palette="Alphabet")
cols <- cols[as.numeric(factor(dat$district))]
funnel(res, col=cols)

Then one can see how points from the same study (or in this case, 'district') cluster together.

To what extent such a plot is indicative of publication bias / small study effects is a different issue (but the same applies even to simpler meta-analyses with a single estimate per study).

Best,
Wolfgang