Dear all, I have a very simple question but, have been having a hard time to figure it out. I am using a mixed model with random intercept and slope using lme function with an unstructured covariance matrix. I know lmer uses Satterthwaite's approximation method to approximate dfs of fixed effects, but I am not sure what is the preferred method that lme uses. Is it Wald or Likelihood ratio? I don't think lme offers such an option to specify an approximation method for dfs of fixed effects. Does it? I appreciate any response in advance. Sala ************* Salahadin (Sala) Lotfi PhD Candidate of Cognitive Neuroscience University of Wisconsin-Milwaukee Anxiety Disorders Laboratory President, Association of Clinical and Cognitive Neuroscience, UWM
lme approximation method for dfs
4 messages · Salahadin Lotfi, Phillip Alday, Maarten Jung +1 more
Hi,
On 23/5/20 9:11 am, Salahadin Lotfi wrote:
Dear all, I have a very simple question but, have been having a hard time to figure it out. I am using a mixed model with random intercept and slope using lme function with an unstructured covariance matrix. I know lmer uses Satterthwaite's approximation method to approximate dfs of fixed effects,
This is not accurate. lme4 by default doesn't even try to figure out the df and doesn't report p-values. The lmerTest package adds in options to use Satterthwaite or Kenward-Roger approximations for p-values, but depending on who you ask around here, the sentiment for those approximations ranges from "of course" to "hmrpf, why would you bother?" to "the heretics must be purged!".? The GLMM FAQ (https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html) has some info on each of these, but I'll copy and paste something relevant that I wrote on a different mailing list: Treating the t-values as z-values is as reasonable as using the t-distribution with some estimated degrees of freedom for studies with 20-30 subjects and 10s of observations per condition per subject for two reasons. One is that a t-distribution with dozens of degrees of freedom is essentially a normal distribution, and so even if you could figure out what the "right" number of degrees of freedom were, it wouldn't be far off from the number you get from the normal distribution. The other reason is that none of these asymptotic results are guaranteed to be particularly great for anything other than very well behaved linear mixed models, which is why things like parametric bootstrap are the gold standard for figuring out coverage intervals. And for large models, bootstrapping is about as fast as KR (because KR as implemented in pbkrmodcomp, which lmerTest depends on, computes the inverse of a large n x matrix).
but I am not sure what is the preferred method that lme uses. Is it Wald or Likelihood ratio?
Wald and likelihood ratio are not degrees of freedom estimates. The likelihood-ratio tests do have a df, which corresponds to the difference in the number of free parameters between the models, but this not the relevant df. (It's numerator degrees of freedom in the ANOVA framework, while what you need are the denominator degrees of freedom.) The Wald tests are just the things you see in the table of the fixed effects, i.e. the tests corresponding to the t- or z-values (or more generally the ANOVA-style tests / tests of linear hypotheses you then construct from the fixed effects).
I don't think lme offers such an option to specify an approximation method for dfs of fixed effects. Does it?
The dfs in nlme are computed using the "inner-outer" rule which doesn't work well for many types of designs common in cognitive neuroscience. More information on this is in the GLMM FAQ, search for "Df alternatives" on that page. Hope that helps! Phillip
I appreciate any response in advance. Sala ************* Salahadin (Sala) Lotfi PhD Candidate of Cognitive Neuroscience University of Wisconsin-Milwaukee Anxiety Disorders Laboratory President, Association of Clinical and Cognitive Neuroscience, UWM [[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
Dear Sala, As Phillip Alday explained, there is no implementation of the Satterthwaite approximation in the nlme package. If you want to stick with this package, the only way I know to get something similar (for lme objects) is to use functions of the emmeans package with the argument "mode" set to "appx-satterthwaite" (see [1]). [1] https://cran.r-project.org/web/packages/emmeans/vignettes/models.html#K Best, Maarten
On Sat, 23 May 2020, 17:07 Phillip Alday <phillip.alday at mpi.nl> wrote:
Hi, On 23/5/20 9:11 am, Salahadin Lotfi wrote:
Dear all, I have a very simple question but, have been having a hard time to figure it out. I am using a mixed model with random intercept and slope using lme
function
with an unstructured covariance matrix. I know lmer uses Satterthwaite's approximation method to approximate dfs of fixed effects,
This is not accurate. lme4 by default doesn't even try to figure out the df and doesn't report p-values. The lmerTest package adds in options to use Satterthwaite or Kenward-Roger approximations for p-values, but depending on who you ask around here, the sentiment for those approximations ranges from "of course" to "hmrpf, why would you bother?" to "the heretics must be purged!". The GLMM FAQ (https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html) has some info on each of these, but I'll copy and paste something relevant that I wrote on a different mailing list: Treating the t-values as z-values is as reasonable as using the t-distribution with some estimated degrees of freedom for studies with 20-30 subjects and 10s of observations per condition per subject for two reasons. One is that a t-distribution with dozens of degrees of freedom is essentially a normal distribution, and so even if you could figure out what the "right" number of degrees of freedom were, it wouldn't be far off from the number you get from the normal distribution. The other reason is that none of these asymptotic results are guaranteed to be particularly great for anything other than very well behaved linear mixed models, which is why things like parametric bootstrap are the gold standard for figuring out coverage intervals. And for large models, bootstrapping is about as fast as KR (because KR as implemented in pbkrmodcomp, which lmerTest depends on, computes the inverse of a large n x matrix).
but I am not sure what is the preferred method that lme uses. Is it Wald or Likelihood
ratio? Wald and likelihood ratio are not degrees of freedom estimates. The likelihood-ratio tests do have a df, which corresponds to the difference in the number of free parameters between the models, but this not the relevant df. (It's numerator degrees of freedom in the ANOVA framework, while what you need are the denominator degrees of freedom.) The Wald tests are just the things you see in the table of the fixed effects, i.e. the tests corresponding to the t- or z-values (or more generally the ANOVA-style tests / tests of linear hypotheses you then construct from the fixed effects).
I don't think lme offers such an option to specify an approximation
method
for dfs of fixed effects. Does it?
The dfs in nlme are computed using the "inner-outer" rule which doesn't work well for many types of designs common in cognitive neuroscience. More information on this is in the GLMM FAQ, search for "Df alternatives" on that page. Hope that helps! Phillip
I appreciate any response in advance.
Sala
*************
Salahadin (Sala) Lotfi
PhD Candidate of Cognitive Neuroscience
University of Wisconsin-Milwaukee
Anxiety Disorders Laboratory
President, Association of Clinical and Cognitive Neuroscience, UWM
[[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
Dear Sala, you could use the parameters package (https://easystats.github.io/parameters), which provides functions to extract degrees of freedom, p-values, or in general summaries of model parameters (see example below), which also gives you Satterthwaite approximated degrees of freedom for models from package nlme. Internally, the lavaSearch2 package is used to calculate the degrees of freedom. Note, however, that the approximated degrees of freedom slightly differ from those that are given by lmerTest::lmer(). This *might* be due to different default settings in optimization etc. between nlme::lme() and lme4::lmer(). Best Daniel library(parameters) library(nlme) data(iris) m1 <- lme( Sepal.Length ~ Petal.Width * Petal.Length + Sepal.Width, data = iris, random = ~ 1 | Species ) model_parameters(m1) #> Parameter | Coefficient | SE | 95% CI | t | df | p #> ---------------------------------------------------------------------------- ----------- #> (Intercept) | 2.33 | 0.37 | [ 1.59, 3.06] | 6.27 | 143 | < .001 #> Petal.Width | -0.93 | 0.24 | [-1.41, -0.45] | -3.83 | 143 | < .001 #> Petal.Length | 0.61 | 0.09 | [ 0.44, 0.78] | 7.14 | 143 | < .001 #> Sepal.Width | 0.56 | 0.08 | [ 0.40, 0.71] | 7.15 | 143 | < .001 #> Petal.Width * Petal.Length | 0.11 | 0.05 | [ 0.01, 0.21] | 2.21 | 143 | 0.029 model_parameters(m1, df_method = "satterthwaite") #> Warning in sCorrect.lme(x, ..., adjust.Omega = value, adjust.n = value): Small sample corrections were derived for ML not for REML #> Parameter | Coefficient | SE | 95% CI | t | df | p #> ---------------------------------------------------------------------------- -------------- #> (Intercept) | 2.33 | 0.37 | [ 1.46, 3.20] | 6.27 | 38.44 | < .001 #> Petal.Width | -0.93 | 0.24 | [-1.52, -0.35] | -3.83 | 46.10 | < .001 #> Petal.Length | 0.61 | 0.09 | [ 0.40, 0.82] | 7.14 | 35.46 | < .001 #> Sepal.Width | 0.56 | 0.08 | [ 0.39, 0.73] | 7.15 | 171.78 | < .001 #> Petal.Width * Petal.Length | 0.11 | 0.05 | [-0.01, 0.23] | 2.21 | 66.29 | 0.031 -----Urspr?ngliche Nachricht----- Von: R-sig-mixed-models <r-sig-mixed-models-bounces at r-project.org> Im Auftrag von Maarten Jung Gesendet: Sonntag, 24. Mai 2020 00:01 Cc: Help Mixed Models <r-sig-mixed-models at r-project.org> Betreff: Re: [R-sig-ME] lme approximation method for dfs Dear Sala, As Phillip Alday explained, there is no implementation of the Satterthwaite approximation in the nlme package. If you want to stick with this package, the only way I know to get something similar (for lme objects) is to use functions of the emmeans package with the argument "mode" set to "appx-satterthwaite" (see [1]). [1] https://cran.r-project.org/web/packages/emmeans/vignettes/models.html#K Best, Maarten
On Sat, 23 May 2020, 17:07 Phillip Alday <phillip.alday at mpi.nl> wrote:
Hi, On 23/5/20 9:11 am, Salahadin Lotfi wrote:
Dear all, I have a very simple question but, have been having a hard time to
figure
it out. I am using a mixed model with random intercept and slope using lme
function
with an unstructured covariance matrix. I know lmer uses Satterthwaite's approximation method to approximate dfs of fixed effects,
This is not accurate. lme4 by default doesn't even try to figure out the df and doesn't report p-values. The lmerTest package adds in options to use Satterthwaite or Kenward-Roger approximations for p-values, but depending on who you ask around here, the sentiment for those approximations ranges from "of course" to "hmrpf, why would you bother?" to "the heretics must be purged!". The GLMM FAQ (https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html) has some info on each of these, but I'll copy and paste something relevant that I wrote on a different mailing list: Treating the t-values as z-values is as reasonable as using the t-distribution with some estimated degrees of freedom for studies with 20-30 subjects and 10s of observations per condition per subject for two reasons. One is that a t-distribution with dozens of degrees of freedom is essentially a normal distribution, and so even if you could figure out what the "right" number of degrees of freedom were, it wouldn't be far off from the number you get from the normal distribution. The other reason is that none of these asymptotic results are guaranteed to be particularly great for anything other than very well behaved linear mixed models, which is why things like parametric bootstrap are the gold standard for figuring out coverage intervals. And for large models, bootstrapping is about as fast as KR (because KR as implemented in pbkrmodcomp, which lmerTest depends on, computes the inverse of a large n x matrix).
but I am not sure what is the preferred method that lme uses. Is it Wald or Likelihood
ratio? Wald and likelihood ratio are not degrees of freedom estimates. The likelihood-ratio tests do have a df, which corresponds to the difference in the number of free parameters between the models, but this not the relevant df. (It's numerator degrees of freedom in the ANOVA framework, while what you need are the denominator degrees of freedom.) The Wald tests are just the things you see in the table of the fixed effects, i.e. the tests corresponding to the t- or z-values (or more generally the ANOVA-style tests / tests of linear hypotheses you then construct from the fixed effects).
I don't think lme offers such an option to specify an approximation
method
for dfs of fixed effects. Does it?
The dfs in nlme are computed using the "inner-outer" rule which doesn't work well for many types of designs common in cognitive neuroscience. More information on this is in the GLMM FAQ, search for "Df alternatives" on that page. Hope that helps! Phillip
I appreciate any response in advance.
Sala
*************
Salahadin (Sala) Lotfi
PhD Candidate of Cognitive Neuroscience
University of Wisconsin-Milwaukee
Anxiety Disorders Laboratory
President, Association of Clinical and Cognitive Neuroscience, UWM
[[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models -- _____________________________________________________________________ Universit?tsklinikum Hamburg-Eppendorf; K?rperschaft des ?ffentlichen Rechts; Gerichtsstand: Hamburg | www.uke.de Vorstandsmitglieder: Prof. Dr. Burkhard G?ke (Vorsitzender), Joachim Pr?l?, Prof. Dr. Blanche Schwappach-Pignataro, Marya Verdel _____________________________________________________________________ SAVE PAPER - THINK BEFORE PRINTING