lme approximation method for dfs

Hi Sala,

Please keep the mailing list in cc.

Looks like you don't have the latest emmeans version. In previous versions
"appx-satterthwaite" was termed "boot-satterthwaite" (see [1]). But
according to [1] just "satterthwaite" should also work; however, this
misses the point that the degrees of freedom that emmeans calculates are
*approximations to* the Satterthwaite approximation (again, see [1]).

[1]
https://cran.r-project.org/web/packages/emmeans/vignettes/models.html#K

Best,
Maarten

On Mon, 25 May 2020, 21:18 Salahadin Lotfi <salahadin.lotfi at gmail.com>
wrote:

Hi Maarten,
Thanks again for additional information. I am actually started to have
issues with *joint_tests**() *function. When I ask for  *mode =
"appx-satterthwaite" *, I keep receiving the below error:
Error in match.arg(mode) :   'arg' should be one of ?containment?,
?satterthwaite?, ?boot-satterthwaite?, ?auto?
Precisely, when I use lmer function for fitting my model, it works with
no issues (I don't specify mode for that as sattherthwaite is default with
lmerTest()). But, when I fit the model with lme with an unstructured
covariance matrix (corSymm), I get the error above.
Would you have any input on this?

Thanks very much,
Sala

On Mon, May 25, 2020 at 3:41 AM Maarten Jung <
Maarten.Jung at mailbox.tu-dresden.de> wrote:

Hi Sala,

I'm glad I could help.
Please keep the mailing list in cc.

Just a small addition to my last mail: If you call joint_tests()
directly on your model (and not on an emmGrid object which is returned
by the functions emmeans() or ref_grid()), you also have to pass the
"mode" argument, otherwise you won't get the degrees of freedom you
want (i.e., use joint_tests(lme_fit, mode = "appx-satterthwaite") and
not just joint_tests(lme_fit)).

Best,
Maarten



On Mon, May 25, 2020 at 7:09 AM Salahadin Lotfi
<salahadin.lotfi at gmail.com> wrote:

Maarten,

Both suggestions were great. I used them and they are perfectly

working.

Sala

On Sun, May 24, 2020 at 4:57 AM Maarten Jung <

Maarten.Jung at mailbox.tu-dresden.de> wrote:

Dear Sala,

I *think* the containment method that is mentioned in the "Models
supported by emmeans" vignette is something similar to the containment
degrees of freedom approximation in SAS [1] but if you want to use
this method, you should ask the author of emmeans (Russell Lenth) to
make sure that my guess is correct.

I guess your emmeans() call throws an error because you are missing
the "specs" argument which specifies the predictor variables over
which the estimated marginal means should be calculated. Also, the
last argument is "sigmaAdjust" instead of "adjustSigma" but it
defaults to TRUE anyway. This should work (if the emmeans package is
loaded, otherwise use emmeans:emmeans()):
emmeans(lme_fit, ~ Group * TIME, mode = "appx-satterthwaite",
sigmaAdjust = TRUE)

The function joint_tests() which constructs Type-III-ANOVA-like tables
of all predictors and the function contrast() could be useful for your
analysis, too. See also the nice vignettes here [2].

[1]

https://documentation.sas.com/?docsetId=statug&docsetTarget=statug_glimmix_details38.htm&docsetVersion=15.1&locale=en

[2]  https://cran.r-project.org/web/packages/emmeans/vignettes/

Best,
Maarten


On Sun, May 24, 2020 at 4:55 AM Salahadin Lotfi
<salahadin.lotfi at gmail.com> wrote:

Wow! Such a thorough answer, Phillip. Thanks very much.
I misspoken when I said the lmer would generate p values for fixed

effects (yes it is indeed lmerTest package which does it).

Just to make sure I understand it correctly:
lme uses ML/REML to approximate beta weights of fixed effects,
it uses Wald to approximate t/z values (fixed effects), and
it uses "inner-outer" rule to get denominator of df (ddf) which is

apparently the least optimal method to estimate ddf among other methods
(etc., Satterthwaite's). Lastly, is this "inner-outer" rule the same as
"containment method"?

Maarten,
First of all thank you for the nice pointer. The emmeans is

elegant. But, I could not get it to work. Could you point out what I am
missing in the emmeans function?

  Here is my code:
## Group and Time are categorical IVs with three levels.
lme_fit<- lme(DV~  factor (Group) * factor(TIME), random = ~ TIME |

SubjectID, # factor(Group) * factor(TIME)

                  correlation = corSymm (form = ~ 1 | SubjectID),
                  weights = varIdent (form = ~ 1 | TIME),
                  data = my_data,
                  method= "REML",
                  na.action = "na.omit")
emmeans::emmeans(lme.fit, mode= "appx-satterthwaite", adjustSigma =

TRUE) ## NOT working

I appreciate for taking time and replying.
Sala

*************
Salahadin (Sala) Lotfi

PhD Candidate of Cognitive Neuroscience

University of Wisconsin-Milwaukee

Anxiety Disorders Laboratory

President, Association of Clinical and Cognitive Neuroscience, UWM


On Sat, May 23, 2020 at 5:01 PM Maarten Jung <

Maarten.Jung at mailbox.tu-dresden.de> wrote:

Dear Sala,

As Phillip Alday explained, there is no implementation of the

Satterthwaite approximation in the nlme package. If you want to stick with
this package, the only way I know to get something similar (for lme
objects) is to use functions of the emmeans package with the argument
"mode" set to "appx-satterthwaite" (see [1]).

[1]

https://cran.r-project.org/web/packages/emmeans/vignettes/models.html#K

Best,
Maarten


On Sat, 23 May 2020, 17:07 Phillip Alday <phillip.alday at mpi.nl>

wrote:

Hi,

On 23/5/20 9:11 am, Salahadin Lotfi wrote:

Dear all,
I have a very simple question but, have been having a hard time

to figure

it out.
I am using a mixed model with random intercept and slope using

lme function

with an unstructured covariance matrix. I know lmer uses

Satterthwaite's

approximation method to approximate dfs of fixed effects,

This is not accurate. lme4 by default doesn't even try to figure

out the

df and doesn't report p-values. The lmerTest package adds in

options to

use Satterthwaite or Kenward-Roger approximations for p-values,

but

depending on who you ask around here, the sentiment for those
approximations ranges from "of course" to "hmrpf, why would you

bother?"

to "the heretics must be purged!".

The GLMM FAQ (

https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html)

has some info on each of these, but I'll copy and paste something
relevant that I wrote on a different mailing list:

Treating the t-values as z-values is as reasonable as using the
t-distribution with some estimated degrees of freedom for studies

with

20-30 subjects and 10s of observations per condition per subject

for two

reasons. One is that a t-distribution with dozens of degrees of

freedom

is essentially a normal distribution, and so even if you could

figure

out what the "right" number of degrees of freedom were, it

wouldn't be

far off from the number you get from the normal distribution. The

other

reason is that none of these asymptotic results are guaranteed to

be

particularly great for anything other than very well behaved

linear

mixed models, which is why things like parametric bootstrap are

the gold

standard for figuring out coverage intervals. And for large

models,

bootstrapping is about as fast as KR (because KR as implemented in
pbkrmodcomp, which lmerTest depends on, computes the inverse of a

large

n x matrix).

but I am not sure
what is the preferred method that lme uses. Is it Wald or

Likelihood ratio?

Wald and likelihood ratio are not degrees of freedom estimates.

The

likelihood-ratio tests do have a df, which corresponds to the

difference

in the number of free parameters between the models, but this not

the

relevant df. (It's numerator degrees of freedom in the ANOVA

framework,

while what you need are the denominator degrees of freedom.) The

Wald

tests are just the things you see in the table of the fixed

effects,

i.e. the tests corresponding to the t- or z-values (or more

generally

the ANOVA-style tests / tests of linear hypotheses you then

construct

from the fixed effects).

I don't think lme offers such an option to specify an

approximation method

for dfs of fixed effects. Does it?

The dfs in nlme are computed using the "inner-outer" rule which

doesn't

work well for many types of designs common in cognitive

neuroscience.

More information on this is in the GLMM FAQ, search for "Df
alternatives" on that page.


Hope that helps!

Phillip

I appreciate any response in advance.
Sala


*************
Salahadin (Sala) Lotfi

PhD Candidate of Cognitive Neuroscience

University of Wisconsin-Milwaukee

Anxiety Disorders Laboratory

President, Association of Clinical and Cognitive Neuroscience,

UWM

      [[alternative HTML version deleted]]