Skip to content

nAGQ

8 messages · Poe, John, Dimitris Rizopoulos, Ben Bolker +1 more

Original
Poe, John
# 
Hello all,

I'm getting ready to teach multilevel modeling and am putting together some
simulations to show relative accuracy of PIRLS, Laplace, and various
numbers of quadrature points in lme4 when true random effects distributions
aren't normal. Every bit of intuition I have says that nAGQ=100 should do
better than nAGQ=11 which should be better than Laplace. Every stats
article I've ever read on the subject also agrees with that intuition.
There was some debate over if it actually matters that some solutions are
more accurate but no debate that they are or are not actually more
accurate. But that's not what's showing up.

When I fit the models and predict Empirical Bayes means I look at
histograms and they look as close to identical as possible. When I use KL
Divergence and Gateaux derivatives to test for differences in the
distributions both show very low scores meaning the distributions are very
very similar.

Furthermore, when I tried a multimodal distribution they all did a bad job
of approximation of the true random effect. The exact same bad job.

I feel like I'm taking crazy pills. The only thing I can think that makes
any sense is lme4 is overriding my choices for approximation of the random
effects in the models themselves or the calculation of the EB means is
being done the same way regardless of the model.

Any ideas?
Ben Bolker
# 
Can you give a few more details of your simulations? E.g. response
distribution, mean of the response, cluster size?
On Sat, Jul 6, 2024, 9:52 PM John Poe <jdpoe223 at gmail.com> wrote:

            

      
      
      
    

        

      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Poe, John
    

    

      
      #
    

  


  

      
Sure,

I wrote several different random effects distributions based mostly on
mixtures of normals. The main idea was that I was trying to break anything
that would assume normality of the random effects when trying to
approximate them.

One of the worst cases I could come up with was a random effect
distribution that had two modes surrounding the mean, one mode was for a
normal distribution and one was for a weibull with a long tail. So both
asymmetrical and multimodal.

All of the simulations had 5000 groups with 500 observations per group and
a binary outcome. I wanted to avoid shrinkage problems or distortions from
too few groups.

I used lme4 to fit the models and extract random effects estimates.

        
On Sun, Jul 7, 2024, 2:29?PM Ben Bolker <bbolker at gmail.com> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Ben Bolker
    

    

      
      #
    

  


  

      
In lme4 the agq stuff is only for GLMMs, ie for glmer not lmer. I'm not
sure of the theory in your case ...

        
On Sun, Jul 7, 2024, 3:50 PM John Poe <jdpoe223 at gmail.com> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Poe, John
    

    

      
      #
    

  


  

      
Yes it's using glmer and not lmer. It's comparing Laplace, AGQ= 7, 11, 51,
and 101 quadrature points compared to the true distribution. Laplace and
the lower values of agq should perform poorly because they are banking on
normality. Higher levels of agq should be more accurate

        
On Sun, Jul 7, 2024, 2:58?PM Ben Bolker <bbolker at gmail.com> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Dimitris Rizopoulos
    

    

      
      #
    

  


  

      
As the number of measurements per group increases, the conditional distribution of the random effects given the observed data (i.e., the posterior of the random effects) converges to a normal distribution, even if the marginal distribution of the random effects (prior) is not normal. See some arguments regarding this here for the related class of shared parameter models: https://doi.org/10.1093/biomet/asm087



??
Dimitris Rizopoulos
Professor of Biostatistics
Erasmus University Medical Center
The Netherlands

        

      
      
    

  
  


  


      

    

    

      
      

        


  

        

      Ben Bolker
    

    

      
      #
    

  


  

      
John, try your examples in GLMMadaptive, which has an independent
implementation of AGQ

On Sun, Jul 7, 2024, 4:27 PM Dimitris Rizopoulos <d.rizopoulos at erasmusmc.nl>
wrote:

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        

  
  


  


      

    

    

      
      

        


  

        

      Douglas Bates
    

    

      
      #
    

  


  

      
I agree with Dimitris.  Adaptive Gauss-Hermite quadrature is used to
approximate the integral of the conditional density of a random effect
given the observed data.  We go into some detail about the model and the
derivation of the integral in question in
https://embraceuncertaintybook.com/aGHQ.html.

With 500 binary observations for each of 5000 groups, the integral in
question will be very close to a scaled Gaussian density, and the Laplace
approximation will be more than adequate. I am not surprised that there are
almost no differences between the results from the Laplace approximation
and AGQ of different orders.  Bear in mind that, for high orders, the
weights drop dramatically for evaluations far from the mode of the
conditional distribution (see Fig. C1 and C2 in the above-mentioned book
for the case of nAGQ = 9).  For very large order, the Golub-Welsch
algorithm, which IIRC is the way the weights and abscissae for the
Gauss-Hermite rule are calculated in lme4, the weights for the remote
evaluations are actually zero.

The table of abscissae and weights for nAGQ=31 is enclosed.  You can see
that when you get beyond three or four standard deviations from the mean
(or "six sigma" for the Quality Control crowd) the additional evaluations
have very little effects on the value of the integral.

Is your simulation based on observed data or an actual study or
experiment?  I have never seen cases of that many observations per group,
especially over that number of groups.

        
On Sun, Jul 7, 2024 at 4:26?PM Ben Bolker <bbolker at gmail.com> wrote:

        

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

            

      
      
      
    

        

      
      
    

        
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: nAGQ31.txt
URL: <https://stat.ethz.ch/pipermail/r-sig-mixed-models/attachments/20240707/1f1da261/attachment.txt>