Skip to content

[R-meta] 4-Level analysis in metafor

7 messages · Harris, Jordan L, Wolfgang Viechtbauer, Michael Dewey

Harris, Jordan L
# 
Hi all,

Does rma.mv appropriately account for between- and within-cluster variance for 4 level nested data?

rma.mv(yi=ES, V=sampling_variance, slab=authors, data=Data, random = list(~ 1 | datasource_id/wave_id/study), tdist=TRUE, method="REML")

study_id = included study
datasource = the source of data (e.g., large cohort study or independent samples)
wave_id = the wave of the datasource (i.e., age) from which the study was analyzed

Multiple effect sizes can occur at a given wave in a given data source. Multiple effect sizes also exist in a given study at a given wave. Provided this information, it might be important to nest studies within waves within data sources. I ask because I see that the sigma^2.2. estimate of my output is nearly 0 and I was not sure if this is an accurate reflection of my data or metafor's ability to account for differences at this added level? Should I use the 0 estimate at 2.2 to justify a removal of wave_id from the nesting?

Multivariate Meta-Analysis Model (k = 100; method: REML)

Variance Components:

            estim    sqrt  nlvls  fixed                          factor
sigma^2.1  0.0069  0.0832     41     no                   datasource_id
sigma^2.2  0.0000  0.0000     60     no           datasource_id/wave_id
sigma^2.3  0.0023  0.0482     82     no  datasource_id/wave_id/study_id

I am a graduate student, and I am new to meta-analyses, and I would love any feedback!
Thanks,
Jordan
Wolfgang Viechtbauer
# 
Hi Jordan,

Sure it can. We have done 5-level models (including another crossed random effect) with rma.mv():

https://wviechtb.github.io/metadat/reference/dat.mccurdy2020.html

How well the variance components can be estimated depends of course on how much data you have. And it can certainly happen that one components ends up being estimated to be (close to) zero.

I wouldn't bother removing that one level - that happens implicitly/automatically when a variance component is estimated to be 0.

Best,
Wolfgang
Harris, Jordan L
# 
Hi Wolfgang,

Thank you very much for the reply!
Do you suggest any specific method for calculating the I2 variance between levels? I found a github package "dmetar" that allows for this calculation for 3-level, but will not allow for calculations greater than 3.

Thanks,
Jordan
Wolfgang Viechtbauer
# 
See: https://www.metafor-project.org/doku.php/tips:i2_multilevel_multivariate

Best,
Wolfgang
Michael Dewey
# 
Dear Jordan

I know this is not what you asked but are you sure that all of those 
should be random effects? Do you not want to fit age as a fixed effect 
as a potential moderator? I also wonder about datasource.

Michael
On 04/03/2022 16:29, Harris, Jordan L wrote:

  
    
  


  


      

    

    

      
      

        


  

        

      Harris, Jordan L
    

    

      
      #
    

  


  

      
Hi Michael,

I am fitting age as a moderator because my primary research question is to see whether the effect size changes with age, I just did not send along my meta-regression. As I see it, datasource would be a random effect because there were 36 distinct datasources and each had different age ranges (i.e., wave). If you think it would be important to consider datasource as a fixed effect moderator, I will happily give it a chance, I just am not sure about what that would mean theroetically.

Jordan

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Michael Dewey
    

    

      
      #
    

  


  

      
Going by the two examples you gave of datasources I assumed something 
different. I agree that is not a fixed effect.

        
On 05/03/2022 16:07, Harris, Jordan L wrote: