Replicating type III anova tests for glmer/GLMM
In my experience, car::Anova is slightly less conservative (as Wald tests are known to be somewhat anti-conservative). Are you using Type-III tests for everything? The differences between Type-II and Type-III can actually make a big difference in terms of which predictors are significant. Speaking of Type-III -- although it's the default in some popular commercial packages, Type-II (marginal tests) is actually the type that makes the most sense in terms of statistical interpretation and hypotheses tested. But that's a topic for another time .... Best, Phillip
On 23/02/16 22:41, Francesco Romano wrote:
Thanks to Henrik and Phillip for the quick reply. Your suggestions have been helpful in making progress. On the one hand Henrik is right about reporting coefficients and standard errors when there are only two levels for the each predictor. This is consistent with two of the sources I mentioned so far. I infer that the authors reported directly from the summary(m1) after use of the mixed function (not car::Anova which yields chi square tests). On the other hand, I don't understand how Cai et al. (2012) p.842, "combined analysis experiments 1 and 2", reported the main effect of a factor with 4 levels via a single estimate, SE, z, p coefficient. How did they obtain this and is this the right way? Finally, after running analysis both ways, I get slightly different p-values, with the car::Anova method being more conservative (it yields less significant predictors). Is this normal? Frank On Tue, Feb 23, 2016 at 10:51 AM, Phillip Alday <Phillip.Alday at unisa.edu.au> wrote:
lme4:anova() is not the same thing as car::Anova()! A quick R note that might have avoided the confusion: The :: syntax in R refers to scope, so you can specify a function unambiguously via package::function.name(). Moreover, R is case sensitive, so Anova() and anova() are generally different things. Henrik's message (posted to the list so if you don't suscribe, you need to look here: https://mailman.stat.ethz.ch/pipermail/r-sig-mixed-models/2016q1/024465.html ) describes how to do this with either his afex package (for likelihood-ratio tests) or John Fox's car package (for analysis of deviance / Wald tests). If you just want to perform likelihood-ratio tests in lme4, then you should look at the drop1() function or you can use anova(reduced.model, full.model). Henrik also does a nice job summarizing some of the issues here, so I won't repeat them. One final note: not everything that holds for normal LMM holds for GLMM -- GLMM tends to be much more complicated. :-( Best, Phillip On 23/02/16 20:03, Francesco Romano wrote:
Yes. An ANOVA with my final bglmer model yields:
anova(recallmodel4x6a)
Analysis of Variance Table
Df Sum Sq Mean Sq F value
syntax12 1 1.7670 1.7670 1.7670
animacy12 1 3.4036 3.4036 3.4036
group123 2 5.7213 2.8607 2.8607
animacy12:group123 2 4.5546 2.2773 2.2773
syntax12:group123 2 8.1732 4.0866 4.0866
which is counterintuitively not what the authors of the papers
apparently used to generate coefficients to report their main effects
and interactions. It looks to me more like ML fitting. Elsewhere,
and more typically, main effects and interactions are obtained by
comparing a
model with the main fixed effect to a model without the
main fixed effect in terms of log-likelihood ratio tests
(Raffray et al., 2013, http://dx.doi.org/10.1016/j.jml.2013.09.004,
p.6).
I understand obtaining p-values from a summary of linear mixed models fit by lmer is a contentious issue https://stat.ethz.ch/pipermail/r-help/2006-May/094765.html but I guess I might be missing something here. On Tue, Feb 23, 2016 at 2:21 AM, Phillip Alday <Phillip.Alday at unisa.edu.au <mailto:Phillip.Alday at unisa.edu.au>> wrote: Have you looked at car::Anova() ? Best, Phillip [forgot to cc the list]
> On 23 Feb 2016, at 11:42, Francesco Romano <
francescobryanromano at gmail.com
<mailto:francescobryanromano at gmail.com>> wrote:
>
> Dear all,
>
> I'm trying to report my analysis replicating the method in the
following
> papers:
>
> Cai, Pickering, and Branigan (2012). Mapping concepts to syntax:
Evidence
> from structural priming in Mandarin Chinese. Journal of Memory and
Language 66
> (2012) 833?849 <tel:%282012%29%20833%E2%80%93849>. (looking at pg.
842, "Combined analysis of Experiments 1
> and 2" section)
>
> Filiaci, Sorace, and Carreiras (2013). Anaphoric biases of null
and overt
> subjects in Italian and Spanish: a cross-linguistic comparison.
Language,
> Cognition, and Neuroscience DOI:10.1080/01690965.2013.801502
(looking at
> pg.11, first two paragraphs)
>
> This is because I have a glmer model with three fixed effects, two
random
> intercepts modeling a binary outcome, exactly as in the articles
mentioned.
>
> The difficulty I'm finding is with locating information on commands
> generating coefficients, SE, z, and p values (e.g. maximum
likelihood
> (Laplace Approximation)) to report main effects and interactions
with the
> anova or afex:mixed commands, following application of effect
coding. I
> have looked in several places, including Ben Bolker's FAQ
> http://glmm.wikidot.com/faq and past posts on the topic in this
r-sig.
> Although there appears to be a plethora of material for lmer, I
can't seem
> to locate anything in the right direction for glmer.
>
> Many thanks for any help.
>
>
>
>
> --
> Frank Romano Ph.D.
>
> *LinkedIn*
> https://it.linkedin.com/pub/francesco-bryan-romano/33/1/162
>
> *Academia.edu*
> https://sheffield.academia.edu/FrancescoRomano
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-mixed-models at r-project.org
<mailto:R-sig-mixed-models at r-project.org> mailing list
-- Frank Romano Ph.D. Tel. +39 3911639149 /LinkedIn/ https://it.linkedin.com/pub/francesco-bryan-romano/33/1/162 /Academia.edu/ https://sheffield.academia.edu/FrancescoRomano