Skip to content

[R-meta] Model with intercept gives 0 heterogeneity but without intercept is ok

7 messages · Luke Martinez, Wolfgang Viechtbauer

Original
Luke Martinez
# 
Dear Colleagues.

I fitted two exact same models except that for one I included the intercept
(Model 1) in the model, for the other, I didn't (Model 2).

I wonder why for Model 1 the estimate of between-study heterogeneity is "0"
but for Model 2 that estimate is not "0"?

Thank you very much,
Luke

set.seed(132)
data <- expand.grid(study = 1:60, outcome = rep(1:2,2))
data$X <- rnorm(nrow(data))
e <- rnorm(nrow(data))
data$yi <- .8+.6*data$X + e
data$vi <- runif(nrow(data))

Model1 <- rma.mv(yi ~ 1 + X, vi, random = ~ 1 | study/outcome, data = dat)

                       estim    sqrt  nlvls  fixed         factor
sigma^2.1  0.0000  0.0001     60     no          study
sigma^2.2  0.4707  0.6861    120     no  study/outcome


Model2 <- rma.mv(yi ~ 0 + X, vi, random = ~ 1 | study/outcome, data = dat)

                    estim    sqrt  nlvls  fixed         factor
sigma^2.1  0.5634  0.7506     60     no          study
sigma^2.2  0.4878  0.6984    120     no  study/outcome
Wolfgang Viechtbauer
# 
Dear Luke,

If X is a continuous moderator, removing the intercept forces the line to go through the origin. That is very rarely a sensible thing to do. See also:

https://www.metafor-project.org/doku.php/tips:models_with_or_without_intercept

Best,
Wolfgang
Luke Martinez
# 
Dear Wolfgang,

Thank you.

1- To make sure I understand this correctly, you're saying that because I
killed the intercept, then the intercept for one or more
continuous moderators equals 0 for all studies, thus, there is no intercept
to vary across the levels of study, hence no between-study variance
component can be estimated (sigma^2.1 == 0), correct?

2- Under this circumstance (killing intercept with continuous moderators
only), the intercepts (or averages) for "study/outcome" combinations can
still vary across study-outcome combinations, and thus, in isolation from
"sigma^2.1",  the other "sigma^2.2" can be [correctly] estimated, correct?

3- I have seen models where the intercept is killed in the fixed part, but
present in the random part. Based on what you said, in such models at least
1 categorical moderator must be present so the between-study variance
component can be estimated (e.g., below), correct?

data$gender <- sample(c("M","F"),nrow(data),replace = TRUE)

rma.mv(yi ~ 0 + X + gender, vi, random = ~ 1 | study/outcome)

                     estim    sqrt  nlvls  fixed         factor
sigma^2.1  0.0000  0.0001     60     no          study  <---   Still "0" ?
sigma^2.2  0.5932  0.7702    120     no  study/outcome

Thank you very much,
Luke

On Mon, Aug 30, 2021 at 11:26 AM Viechtbauer, Wolfgang (SP) <
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

            

      
      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Wolfgang Viechtbauer
    

    

      
      #
    

  


  

          

      
      
      
    

        
This doesn't sound right. Removing the intercept in this model says that the average effect must be 0 when X = 0. One can still estimates the variance components whether there is an intercept or not. They have different interpretations though, since the variances are estimated as deviations from a line that has an intercept of 0 or not.

            

      
      
      
    

        
Again, both variance components can be estimated. It just happens to be the case that sigma^2.1 is estimated to be essentially 0 in Model1.

            

      
      
      
    

        
Again, all variance components can be estimated whether the fixed part includes an intercept or not.

            

      
      
      
    

        
In this case, the model is identical whether you use '0 + X + gender' or 'X + gender', just the parameterization of the fixed part is different. Please see the link I posted.

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Luke Martinez
    

    

      
      #
    

  


  

      
Thank you, Wolfgang. I visited the link you kindly shared. But that link
only discusses the effect of removing the intercept on the fixed parts, not
random parts.

Also, in that link the fixed parts only include either a categorical or
only a continuous moderator, but not both types of moderators together. For
example, if we have two categorical moderators and one
continuous moderator, as in:

data$gender <- sample(c("M","F"),nrow(data),replace = TRUE)
data$sector <- sample(c("Pr","Pv", "NGO"),nrow(data),replace = TRUE)

Then, removing the intercept is the matter of which categorical moderator
appears last in the formula! For example, in:

(A): rma.mv(yi ~  0 + gender + sector +  X , vi, random = ~ 1 |
study/outcome, data = data)

R removes the intercept for "sector" because it appears last. But, in:

(B): rma.mv(yi ~  0 + sector + gender +  X , vi, random = ~ 1 |
study/outcome, data = data)

R removes the intercept for "gender" because it appears last.

My question is that do these behaviors in the fixed-part, essentially,
change the meaning/nature (e.g., what average is varying across study
levels) of the random parts?

Apparently, the random part is not related to the fixed-part in rma.mv(),
and that's why both (A) and (B), with or without the intercepts (i.e., 4
specification) all give the exact same estimates of their two variance
components?

Thank you,
Luke



On Mon, Aug 30, 2021 at 2:43 PM Viechtbauer, Wolfgang (SP) <

        
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote:

        

            

      
      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Wolfgang Viechtbauer
    

    

      
      #
    

  


  

          

      
      
      
    

        
In the models above: No. All of the models are above are essentially the same, the fixed effects are just parameterized differently. They have the same log likelihood, the same fitted values, the same residuals, etc. etc.

            

      
      
      
    

        
This is true if the different parameterizations of the fixed effects are ultimately identical (as is the case above).

            

      
      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Luke Martinez
    

    

      
      #
    

  


  

      
Thank you. That would mean that in all four cases below, I can
interpret sigma^2.1 as: the variation in true effect sizes at the study
level (i.e., averaged within each study across all studies), above and
beyond the explanatory power of gender, sector, and X (if such variables
across studies).

Thanks, again,
Luke

(A): rma.mv(yi ~  0 + gender + sector +  X , vi, random = ~ 1 |
study/outcome, data = data)
(B): rma.mv(yi ~  0 + sector + gender +  X , vi, random = ~ 1 |
study/outcome, data = data)
(C): rma.mv(yi ~  gender + sector +  X , vi, random = ~ 1 | study/outcome,
data = data)
(D): rma.mv(yi ~  sector + gender +  X , vi, random = ~ 1 | study/outcome,
data = data)

On Mon, Aug 30, 2021 at 4:54 PM Viechtbauer, Wolfgang (SP) <

        
wolfgang.viechtbauer at maastrichtuniversity.nl> wrote: