Skip to content

[R-meta] Multilevel / Specification of levels

6 messages · Dudenhöffer, Tina, Michael Dewey, Wolfgang Viechtbauer +1 more

Dudenhöffer, Tina
# 
Dear all,

I am running a multilevel model using the rma.mv function.

res4 <- rma.mv(Hedges_g, Hedges_v, method=?REML", random = list( ~ 1 | Study), data=Data)
res4

The question I have is regarding the random argument. If I understand correctly, this is where the levels are specified. My effect sizes (Hedges_g) are nested in studies (Study). The variable Study refers to the unique study ID in my data set. When reading the documentation, it says that the ~ 1 | id is a grouping id and outcomes with the same id receive the same random effect. Hence, in my example, all outcomes from the same study would receive the same random effect.

However, I have also seen other models that seem to include the unique effect size ID and if I translate this to my example it would look like this:

res4 <- rma.mv(Hedges_g, Hedges_v,  method=?REML", random = list(~ 1 | EScumulative, ~ 1 | Study), data=Data)
res4

As you can see the difference is the ~ 1 | EScumulative. EScumulative in my data set refers to each unique effect size. Based on the documentation and the notion that the id in the random argument refers to grouping variables, I don?t believe that I should include the EScumulative in the model, because none of them share the same value (unlike the id for Study, where some effect sizes share the same study ID. However, there are some examples (and opinions) that suggest otherwise.

That said: My question is, which of the two models is correct?

I am now working using the second model (including the EScumulative):

The variance components  look like this: so to me this almost seems like there is no variation on the study level and I wonder if I should even run multilevel?

                   estim    sqrt      nlvls  fixed        factor
sigma^2.1     0.2283  0.4778     62     no  EffectSizeID
sigma^2.2     0.0000  0.0000     28     no         Study


I am new to Meta Analysis, new to Multilevel and new to R - excuses in case some of this is not clear.

Thanks for your help and kind regards,

Tina
Michael Dewey
# 
Dear Tina

My understanding of this is that you do need to specify a random term 
for each effect as well to get the equivalent of a random effects 
meta-analysis. Do doubt someone more expert than I will chip in here if 
that is not correct.

Michael
On 17/09/2019 10:55, Dudenh?ffer, Tina wrote:

  
    
  


  


      

    

    

      
      

        


  

        

      Wolfgang Viechtbauer
    

    

      
      #
    

  


  

      
Michael is absolutely right. You should include the row/estimate-level random effect in the model. See also:

http://www.metafor-project.org/doku.php/analyses:konstantopoulos2011#a_common_mistake_in_the_three-level_model

Best,
Wolfgang

-----Original Message-----
From: R-sig-meta-analysis [mailto:r-sig-meta-analysis-bounces at r-project.org] On Behalf Of Michael Dewey
Sent: Wednesday, 18 September, 2019 9:47
To: Dudenh?ffer, Tina; r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Multilevel / Specification of levels

Dear Tina

My understanding of this is that you do need to specify a random term 
for each effect as well to get the equivalent of a random effects 
meta-analysis. Do doubt someone more expert than I will chip in here if 
that is not correct.

Michael

        
On 17/09/2019 10:55, Dudenh?ffer, Tina wrote:

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Dudenhöffer, Tina
    

    

      
      #
    

  


  

      
Dear Michael & Wolfgang,

thank you very much for the quick response.

In the meantime I, indeed, proceeded with specifying random effects for each effect size (that is, I included the EScumulative in the model)

I am still stripping over the definition "~ 1 | id is a grouping id? because each effect size is unique and hence there is no grouping ID for those. However, based on your comments, I am glad to see I am specifying the model correctly, by including each effect size.

Many thanks and kind regards,
Tina

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Wolfgang Viechtbauer
    

    

      
      #
    

  


  

      
Hi Tina,

A group can consist of one member, which is the case here. So, 'id' can also be unique for every row in the dataset. In fact, this is what we do in the standard random-effects model. See: 

http://www.metafor-project.org/doku.php/tips:rma.uni_vs_rma.mv

The model you are using then *adds* random effects at the study level.

Best,
Wolfgang

-----Original Message-----
From: Dudenh?ffer, Tina [mailto:t.dudenhoffer at uva.nl] 
Sent: Wednesday, 18 September, 2019 15:40
To: Viechtbauer, Wolfgang (SP)
Cc: Michael Dewey; r-sig-meta-analysis at r-project.org
Subject: Re: [R-meta] Multilevel / Specification of levels

Dear Michael & Wolfgang,

thank you very much for the quick response.

In the meantime I, indeed, proceeded with specifying random effects for each effect size (that is, I included the EScumulative in the model)

I am still stripping over the definition "~ 1 | id is a grouping id? because each effect size is unique and hence there is no grouping ID for those. However, based on your comments, I am glad to see I am specifying the model correctly, by including each effect size.

Many thanks and kind regards,
Tina

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Tina Dudenhoeffer
    

    

      
      #
    

  


  

      
All clear now.

Thank you!