Skip to content
Prev 16013 / 20628 Next
Original Raw

model specification in lmer

Paul Johnson 
Hi Silvia & CC,
should be
I didn?t know you could test a random effect with anova(lmer(?, REML = FALSE), lm(?)). Is that valid when fitted with ML? 

I recommend reading http://bbolker.github.io/mixedmodels-misc/glmmFAQ.html :

"With recent versions of lme4, goodness-of-fit (deviance) can be compared between (g)lmer and (g)lm models, although anova() must be called with the mixed ((g)lmer) model listed first. Keep in mind that LRT-based null hypothesis tests are conservative when the null value (such as ?2=0) is on the boundary of the feasible space; in the simplest case (single random effect variance), the p-value is approximately twice as large as it should be (Pinheiro and Bates 2000).
	? Consider not testing the significance of random effects. If the random effect is part of the experimental design, this procedure may be considered ?sacrificial pseudoreplication? (Hurlbert 1984). Using stepwise approaches to eliminate non-significant terms in order to squeeze more significance out of the remaining terms is dangerous in any case.
	? consider using the RLRsim package, which has a fast implementation of simulation-based tests of null hypotheses about zero variances, for simple tests. (However, it only applies to lmer models, and is a bit tricky to use for more complex models.)"
To avoid confusion between random and fixed effects, it?s better not to call ?(Trt|Family)" an interaction (even though it has a lot in common with an interaction). Pop:Trt is an interaction. The difference between (1|Family) and (Trt|Family) is that in the former only the intercept varies randomly between families, while in the latter the main effect of Trt also varies randomly between families. The null hypothesis being tested is that both the variance of the Trt effect and its covariance with the random intercept are zero.

Another simple way to test for a random effect in a lmer fit (if you still want to despite the advice quoted above) would be to use confint(fit) and see if the 95% CI for the family variance includes zero. I did a very quick-and-dirty permutation-based simulation to compare confint(?, method = "profile") with the anova(lmer, lm) test:

library(lme4)
library(nlme)
sim.res <- 
  replicate(1000, {
    Orthodont$Subject <- sample(Orthodont$Subject)
    fm1 <- lmer(distance ~ age + (1 | Subject), data = Orthodont, REML = FALSE)
    fm1.lm <- lm(distance ~ age, data = Orthodont)
    c(LRT = anova(fm1, fm1.lm)[2, "Pr(>Chisq)"] < 0.05, 
      CI = confint(fm1, 1)[1] > 0)
  })
table(sim.res["LRT", ], sim.res["CI", ])

The results were conservative (false positive rate of ~2%), which I expected for the LRT, but not for the CI, and completely concordant, presumably because both use the likelihood:
       
        FALSE TRUE
  FALSE   982    0
  TRUE      0   18

I ran it a few more times and it was very consistent, ~2% false positives and 100% concordance.

Best wishes,
Paul

Thread (4 messages)

Paul Johnson model specification in lmer Nov 30 Paul Johnson model specification in lmer Dec 1 silvia.matesanzgarcia at gmail.com model specification in lmer Dec 1 Paul Johnson model specification in lmer Dec 1