Skip to content

glmer and nAGQ

6 messages · Peter R Law, Ben Bolker

Original
Peter R Law
# 
I am re-sending this query that I originally emailed June 13'th. I did not receive a copy of the sent email as I have with previous postings and my query doesn't appear in the archive of postings so does not seem to have been received.

Peter R Law

Sent with [ProtonMail](https://protonmail.com/) Secure Email.

??????? Original Message ???????
On Monday, June 14th, 2021 at 10:48 PM, Peter R Law <prldb at protonmail.com> wrote:

            

      
      
      
    

  
  


  

    

      Message-ID:
      <eRi5lAvI3CZFpoVa8IsTe5tGL_XcunhS6SO19GpY3rHle2EAxwTwqPoXdjNc08xqm-V6bHJFOWwEMh5uGETjT0_tZXZjbp_wVZ02j2HtckI=@protonmail.com>
    

    

      

        
          
            
          
          Original archive
        
        
          
            
            
          
          View message
        
      

      

        1 / 6
        
          
        
      

    

  



    

    

      
      

        


  

        

      Ben Bolker
    

    

      
      #
    

  


  

      
This is kind of hard to read, but I'll try.

   First of all, log-likelihoods with nAGQ>1 (adaptive Gauss-Hermite 
quadrature) **are not currently commensurate with log-likelihoods with 
nAGQ==1 (Laplace approximation). This is hinted at in the "Deviance and 
log-likelihood of GLMMs" section under "Deviance and log-likelihood of 
GLMMs".

   In any case, the log-likelihoods computed under different numbers of 
quadrature points aren't comparable as though they were different 
statistical models (AIC, likelihood ratio tests, etc.). Rather, they are 
based on **different approximations** to the same model, and we know 
that larger numbers of quadrature points (increasing nAGQ) are *more 
accurate* (although slower) approximations.  The main thing is to see if 
*results* (parameter estimates etc.) change as nAGQ increases by a 
magnitude that is important for the current analysis ... if they do, 
then you should use an nAGQ that is large enough that it approximates 
"infinity", i.e. increasing nAGQ further doesn't change the answers by 
an important amount.

   In general if it is not important to you to use the inverse link (for 
interpretation/because you think this will be the natural scale on which 
to measure effects of continuous parameters and/or interactions), 
Gamma(link="log") is generally more robust.

   The "rescale variables?" suggestion is just that; if the parameters 
are already rescaled, then it's not going to help.  A large eigenvalue 
may not actually be a problem, it just indicates the *possibility* of 
numerical instability. In this case my guess is that the very small 
random effects variance might be responsible.

  It's also a little surprising that the normRain coefficient is very 
small in magnitude and yet but has a very large Z-statistic, given this 
sample size.

  The results with increasing AGQ do indeed look kind of wonky.  If this 
were my problem I would want to look more carefully at the data/think 
about where they came from.  How do the model diagnostics look?

        
On 6/18/21 3:52 PM, Peter R Law via R-sig-mixed-models wrote:

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Peter R Law
    

    

      
      #
    

  


  

      
Thanks for your response. I'm sorry the formatting of the R output got messed up and hard to read when I forwarded the original message. I do understand that changing the value of nAGQ is changing the approximation of the likelihood, not changing the model itself. I wondered whether the change in likelihood from -643.8 (n=1) to -5.9 (n=2) to -inf (n=3) was a reason to suspect the model was badly behaved in that none of these approximations of the logLik should be trusted. The parameter estimates for the fixed effects don't change that much, basically just the logLik and consequently the AIC. I wanted to compare the glmer with a Gamma distribution to the lmer to see whether AIC favoured one over the other. My intent was to start with the glmer with default n=1, then increase n until model parameters seemed stable, then use that n value for the glmer model to obtain a logLik and then AIC to compare to the lmer version. But the dramatic differences in logLik in going from n=1 to n= 2 and the negative infinite value already for n=3 I thought might suggest that none of these logLiks should be trusted and the the Gamma model simply abandoned. Does that sound plausible in light of the warning messages? In particular, that the negative infinity answer is a failuer in the estimation procedure rather than an indication that that the Gamma model has zero likelihood?

I will repeat the analyses with the log link. As far as the data is concerned, it's simulated data I created without using any distributional assumptions but rather based on ecological assumptions for how the response might depend on the fixed predictors. My purpose here is to explore the glmer function in preparation for a study with real data (still being collected). Because the responses are positive values, I thought it would be worth investigating how a Gamma glmer compares with an lmer (and compare both with a log-normal model, after making the adjustment in the latter so that its AIC is comparable to the others, as was pointed out in a previous posting).

Peter


Sent with ProtonMail Secure Email.

??????? Original Message ???????

        
On Friday, June 18th, 2021 at 5:44 PM, Ben Bolker <bbolker at gmail.com> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  

    

      Message-ID:
      <CMAdImRcLjL5-85lpWAMiEgGkFecpFON3PRPgT2K0GyVYJzWqpfgCDRttmpyV4R_OMXMHP6zTMTBVOZTjKNI9n6hU77F8_u1XpPboKK6ca8=@protonmail.com>
    

    

      

        
          
            
          
          Original archive
        
        
          
            
            
          
          View message
        
      

      

        
          
        
        3 / 6
        
          
        
      

    

  



      

    

    

      
      

        


  

        

      Ben Bolker
    

    

      
      #
    

  


  

      
On 6/18/21 6:39 PM, Peter R Law wrote:

            

      
      
      
    

        
As I suggested in my answer, you shouldn't worry about the big jump 
in log-likelihood from nAGQ=1 to nAGQ=2. I would try n=5, 10, 15, 20 and 
see how it goes (unless n=20 is too slow to be practical). (Yes, I think 
-Inf is a problem with the fitting procedure)

            

      
      
      
    

        
Fair enough; "without any distributional assumptions" is a bit odd 
though.  Unless your ecological model is stochastic (in which case the 
conditional distribution will emerge naturally from the model), you have 
to make *some* assumptions about how the observations will vary around 
the expected mean ...

   cheers
    Ben Bolker

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  


      

    

    
5 days later

    

      
      

        


  

        

      Peter R Law
    

    

      
      #
    

  


  

      
Thanks again for your feedback. Much appreciated. With the canonical link in the Gamma model, once a neg infinity result is returned for the logLik (n=4 for my simulation data), increasing the value of nAGQ doesn't change that result and the warning message about the estimated variance-covariance matrix not being positive definite is constant also.

If I use the log link, there is a large increase (-678.1 to -5.6) in the LogLik in going from the default value of n=1 to n=2, but it then doesn't change for larger n values and the model results are stable.

So I assume then that the Gamma model with the canonical link cannot be fit to this data while with the log link one does get a robust result?

There was no detectable delay in the time to run the model with nAGQ = 25 versus the default value. Is that suspicious?

Peter



Sent with ProtonMail Secure Email.

??????? Original Message ???????

        
On Friday, June 18th, 2021 at 8:16 PM, Ben Bolker <bbolker at gmail.com> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

  
  


  

    

      Message-ID:
      <LNnAz-OD7BJ9VSGO8RarV2UtO6AP82-oxLiYGqm-CuqYzm5AL_3QN-De_mLduTYGSa49c3CIQmFx5Xp1ELtdx3zSG3zSyRGGE0qQNvf21dc=@protonmail.com>
    

    

      

        
          
            
          
          Original archive
        
        
          
            
            
          
          View message
        
      

      

        
          
        
        5 / 6
        
          
        
      

    

  



      

    

    

      
      

        


  

        

      Ben Bolker
    

    

      
      #
    

  


  

      
It's hard to make many more conclusions without digging into the 
particular data set.

   It might be worth comparing results from other packages, although 
GLMMadaptive is the only other package that I'm aware of that 
robustly/flexibly provides adaptive Gauss-Hermite quadrature.

   I also note lots of 'singular fit' messages below; that doesn't mean 
the fits are wrong, but does mean that you might need to consider 
whether you have enough signal to estimate the random effect variance ...

        
On 6/23/21 10:39 PM, Peter R Law wrote: