[R-meta] Funnel Plots for Multilevel Meta

Dear Michael and Wolfgang.

at the end of Egger's regression / funnel plot thread, Wolfgang said: 
"To what extent such a plot is indicative of publication bias / small 
study effects is a different issue"

Could you please say more on this topic?
Particularly I am interested in this: is a plot with 8 effects informative?
Or a plot with less than 20 effects?

Best,
Valeria

On Fri, Dec 11, 2020 at 11:28 PM Viechtbauer, Wolfgang (SP) 
<wolfgang.viechtbauer at maastrichtuniversity.nl 
<mailto:wolfgang.viechtbauer at maastrichtuniversity.nl>> wrote:

    A funnel plot is simply a plot of the estimates against their
    standard errors (or some other measure of precision). So one can
    draw such a plot whether there are multiple estimates from the same
    study or not. Hence, funnel() in metafor will happily do so:

    library(metafor)

    dat <- dat.konstantopoulos2011
    res <- rma.mv <http://rma.mv>(yi, vi, random = ~ 1 |
    district/school, data=dat)
    res
    funnel(res)

    One could indicate (with different colors or plotting symbols) which
    estimates belong to the same study.

    cols <- palette.colors(length(unique(dat$district)), palette="Alphabet")
    cols <- cols[as.numeric(factor(dat$district))]
    funnel(res, col=cols)

    Then one can see how points from the same study (or in this case,
    'district') cluster together.

    To what extent such a plot is indicative of publication bias / small
    study effects is a different issue (but the same applies even to
    simpler meta-analyses with a single estimate per study).

    Best,
    Wolfgang

     >-----Original Message-----
     >From: R-sig-meta-analysis

    [mailto:r-sig-meta-analysis-bounces at r-project.org
    <mailto:r-sig-meta-analysis-bounces at r-project.org>]

     >On Behalf Of Dylan Johnson
     >Sent: Friday, 11 December, 2020 20:03
     >To: Michael Dewey; r-sig-meta-analysis at r-project.org

    <mailto:r-sig-meta-analysis at r-project.org>

     >Subject: Re: [R-meta] Funnel Plots for Multilevel Meta
     >
     >Hi Michael,
     >
     >Would it not be nonsensical to have multiple effects from the same

    article

     >in the funnel plot though?
     >
     >With the Egger's regression I was able to accommodate the fact

    that their is

     >nonindependence of the effects, but am unsure how to proceed with

    a funnel

     >plot.
     >
     >Dylan
     >
     >Dylan Johnson, MSc
     >
     >MA Student, School and Clinical Child Psychology
     >Department of Applied Psychology and Human Development
     >
     >University of Toronto
     >252 Bloor Street West
     >
     >Toronto, ON M5S 1V6
     >
     >From: Michael Dewey<mailto:lists at dewey.myzen.co.uk

    <mailto:lists at dewey.myzen.co.uk>>

     >Sent: December 11, 2020 6:12 AM
     >To: Dylan Johnson<mailto:dylanr.johnson at mail.utoronto.ca

    <mailto:dylanr.johnson at mail.utoronto.ca>>; r-sig-meta-

     >analysis at r-project.org

    <mailto:analysis at r-project.org><mailto:r-sig-meta-analysis at r-project.org
    <mailto:r-sig-meta-analysis at r-project.org>>

     >Subject: Re: [R-meta] Funnel Plots for Multilevel Meta
     >
     >Dear Dylan
     >
     >Perhaps I misunderstand you but if you have the data for a regression
     >type test like Egger's do you not just plot that? The funnel()

    function

     >in metafor does that and I am sure equivalent solutions can be

    found in

     >meta and many other packages.
     >
     >Michael
     >
     >On 11/12/2020 01:14, Dylan Johnson wrote:

     >> Hello,
     >>
     >> Thanks everyone for helping me sort out the Egger's test with

    multi-level

     >meta modelling!

     >>
     >> Is there any option in R for producing Funnel plots that are

    appropriate

     >for the nonindependence? I imagine that the standard funnel plot

    would be

     >deceiving if it came from a multi-level design.

     >>
     >> Many thanks!
     >>
     >> Best,
     >> Dylan
     >>
     >> Dylan Johnson, MSc
     >>
     >> MA Student, School and Clinical Child Psychology
     >> Department of Applied Psychology and Human Development
     >>
     >> University of Toronto
     >> 252 Bloor Street West
     >>
     >> Toronto, ON M5S 1V6