test significance of single random effect

Dear all,
I am coming back on the recent issue on how to test the significance  
of a single random term in linear mixed models...

In Zuur et al.  "Mixed Models and Extentions in Ecology with R"  
Springer, 2009, the authors suggest to compare a lme model (with the  
random effect) with a gls model with the same fixed effects structure,  
and then compare the AICs of the two models or using a likelihood  
ratio test via the ANOVA comand (pages 122 - 128).

I would be interested in hearing the opinion of other members of the  
list on this approach...

Thanks a lot,

Achaz


On 17 Nov 2009, at 20:41, Tom Van Dooren wrote:

With REML=FALSE RLRsim seems to work fine in R 2.10, if I use the  
design matrix and Zt as arguments in LRTSim().
Otherwise I didn't get useful results out.

That's not too much of a problem.
It is not difficult to simulate the null model without random  
effect, extract logLikelihoods from the (generalized) mixed model  
and the (generalized) linear model fitted to those pseudo-data, to  
calculate a distribution of likelihood ratios,
which are then maybe off by a constant.
What I was mainly uncertain about, is whether the log-likelihood of  
a mixed model (also fitted to data simulated from the null model  
without random effect),
can be used as a statistic itself?
The answer might be a simple NO! of course, or something more  
involved...

Tom


Douglas Bates wrote:

On Tue, Nov 17, 2009 at 3:49 AM, Matthias Gralle
<matthias_gralle at eva.mpg.de> wrote:

I had basically the same problem a short time ago, and resorted to  
lme
instead of lmer, because one can directly compare lme and lm  
objects using
anova(). Is that OK, or is this feature of lme depreciated ?

Is that not possible for linear mixed-effects models fit by lmer  
using
REML = FALSE? (Occasionally I lose track of what can be done in
different versions of lme4.)  You don't want to compare an lmer model
fit by REML with the log-likelihood of an lm model but you should be
able to compare likelihoods (subject to the caveat that the p-value
for the likelihood ratio test on the boundary of the parameter space
is conservative).

Ben Bolker wrote:

Have you tried the RLRsim package??

Tom Van Dooren wrote:

I tried to find an easy way to test whether the random effect  
would be
significant in a (generalized) mixed model with a single random  
effect.
It annoyed me that log-likelihoods of lm or glm and lmer are not
necesarily directly comparable -> trouble with calculating  
likelihood
ratios.
What do members of this list think of the following simulation  
approach?
It basically amounts to simulating a distribution for the log  
likelihood,
given the null hypothesis that there is no random effect  
variance and that
the fixed effect model is correct.


library(lme4)
mm1 <- lmer(Reaction ~ Days + (1|Subject), sleepstudy)
lm1<- lm(Reaction ~ Days, sleepstudy)


LL<-numeric(500)
for(i in 1:500){
resp<-simulate(lm1)
LL[i]<-logLik(lmer(resp[,1] ~ Days + (1|Subject), sleepstudy))
}

hist(LL)
logLik(mm1)
mean(LL>logLik(mm1))

--
Matthias Gralle, PhD
Dept. Evolutionary Genetics
Max Planck Institute for Evolutionary Anthropology
Deutscher Platz 6
04103 Leipzig, Germany
Tel +49 341 3550 519
Fax +49 341 3550 555

Dr. Achaz von Hardenberg
--------------------------------------------------------------------------------------------------------
Centro Studi Fauna Alpina - Alpine Wildlife Research Centre
Servizio Sanitario e della Ricerca Scientifica
Parco Nazionale Gran Paradiso, Degioz, 11, 11010-Valsavarenche (Ao),  
Italy

Present address:
National Centre for Statistical Ecology
School of Mathematics, Statistics and Actuarial Science,
University of Kent,  Canterbury, UK