testing fixed effects in binomial lmer...again?
On Jan 8, 2008 5:38 AM, Achaz von Hardenberg <fauna at pngp.it> wrote:
Dear all,
I know that this may be a already debated topic, but even searching the R-help and the r-sig-mixed-models archives I can not find a reply to my doubts...(but see Ben Bolkers' reply to my similar quest in r- help).
I am performing a binomial glmm analysis using the lmer function in the lme4 package (last release, just downloaded). I am using the "Laplace method".
Yes, that is the best choice in lmer. (In the development version it is, in fact, the only choice.)
However, I am not sure about what I should do to test for the significance of fixed effects in the binomial case: Is it correct to test a full model against a model from which I remove the fixed effect I want to test using the anova(mod1.lmer, mod2.lmer) method and then relying on the model with the lower AIC (or on the Log- likelihood test?)?
The change in the log-likelihood between two nested models is, in my opinion, the most sensible test statistic for comparing the models. However, it is not clear how one should convert this test statistic to a p-value. The use of the chi-squared distribution is based on asymptotic results and can give an "anti-conservative" (i.e. lower than would be obtained through a randomization test or via simulation) p-value for small samples. As far as I can see, the justification for the use of AIC as a comparison criterion is even more vague. For linear fixed-effects models one can compensate for small samples by changing from z-tests to t-tests and from chi-squared tests to F tests. The exact theory breaks down for mixed-effects models or for generalized linear models and is even more questionable for generalized linear mixed models. As Ben Bolker mentioned, I think that one way to deal with the hypothesis testing question while preserving the integrity of the model is to base inferences on a Markov-chain Monte Carlo sample from the (Bayesian) posterior distribution of the parameters. Code for MCMC samples for parameters in GLMMs is not yet fully developed (or documented). In the meantime I would use the likelihood ratio tests but exercise caution in reporting p-values for small-sample cases.
Would you advice me to use the glmmML function instead? (I am not sure where the differences are with lmer) I thank in advance for your help! best regards, Achaz von Hardenberg Ben Bolker wrote:
>The short answer is that testing fixed effects in GLMMs >is difficult and dangerous. Likelihood ratio tests on fixed >effect differences [which is generally what anova() does] >in a random-effects model are unreliable >(see Pinheiro and Bates 2000). Most of the world does >F tests with various corrections on the denominator >degrees of freedom, but this is contentious (in particular, >Doug Bates, the author of lme4, disagrees). lme4 will >eventually let you use an MCMC sampling method to test >fixed effects but that may or may not be working >in the current version.
>I would try this question again on the r-sig-mixed >mailing list.
> good luck, > Ben Bolker
Dr. Achaz von Hardenberg
------------------------------------------------------------------------
--------------------------------
Centro Studi Fauna Alpina - Alpine Wildlife Research Centre
Servizio Sanitario e della Ricerca Scientifica
Parco Nazionale Gran Paradiso, Degioz, 11, 11010-Valsavarenche (Ao),
Italy
------------------------------------------------------------------------
--------------------------------
[[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models