Question about non-significant interactions
---------- Forwarded message --------- From: Francesco Romano <fbromano77 at gmail.com> Date: Tue, Jul 9, 2019 at 11:24 PM Subject: Re: [R-sig-ME] Question about non-significant interactions To: Fox, John <jfox at mcmaster.ca> Dear John, Thanks for the reply. One of my research entails examining the relationship between 3 groups of speakers, the 3 levels of the group categorical variable previously mentioned, and two tasks. One prediction is that one group will perform better than other groups on one test but not the other. I fit a maximal model using the bglmr function as shown previously, then used car::Anova to determine main effects. My understanding from previous interaction with you precisely here on r-sig-me is that the function works as a form of shortcut to the traditional way of model-fitting/ reduction via the function anova() comparing models, eliminating terms one at a time. I hope this is clearer now and yes, the question is more of a statistical one than an R one, even though I suspect the mixed-effect aspect of the regression may be relevant to answering it. Frank
Tue, Jul 9, 2019 at 11:03 PM Fox, John <jfox at mcmaster.ca> wrote:
Dear Francesco, I didn't entirely follow your question and I expect that to answer it, it would be necessary to know more about what your research entails. As you imply, this seems to be more a statistics question than an R question. It's also not clear to me what function you used to fit the mixed-effects logistic regression. But I did notice that you're apparently using Anova() for type-III tests with the default contr.treatment() coding for factors. The main-effect tests that result are not sensible. As it says in ?Anova: "Warning Be careful of type-III tests: For a traditional multifactor ANOVA model with interactions, for example, these tests will normally only be sensible when using contrasts that, for different terms, are orthogonal in the row-basis of the model, such as those produced by contr.sum, contr.poly, or contr.helmert, but not by the default contr.treatment. In a model that contains factors, numeric covariates, and interactions, main-effect tests for factors will be for differences over the origin. In contrast (pun intended), type-II tests are invariant with respect to (full-rank) contrast coding. If you don't understand this issue, then you probably shouldn't use Anova for type-III tests." I hope that this is of some help, John ----------------------------- John Fox, Professor Emeritus McMaster University Hamilton, Ontario Canada L8S 4M4 web: socserv.mcmaster.ca/jfox
________________________________________
From: R-sig-mixed-models [r-sig-mixed-models-bounces at r-project.org] on
behalf of Francesco Romano [fbromano77 at gmail.com]
Sent: July 9, 2019 9:49 AM
To: r-sig-mixed-models at r-project.org
Subject: [R-sig-ME] Question about non-significant interactions
Dear all,
I have more of a theoretical than practical question for you. The model I
am using has two IVs, group (3 levels) and task (2 levels), and a
categorical DV (correct versus incorrect), hence logistic regression.
Random effects for subjects and items, as well as slopes for group by item
and task by subject.
I am interested in the effect of belonging any of three groups, the levels
of the group IV, in order to test some a priori predictions. The bayesian
wrapper is to help the model converge.
Here is the output:
summary(paper2analysis1)
Cov prior : item ~ wishart(df = 5.5, scale = Inf, posterior.scale = cov,
common.scale = TRUE)
: Participant ~ wishart(df = 4.5, scale = Inf, posterior.scale =
cov, common.scale = TRUE)
Prior dev : 6.9466
Generalized linear mixed model fit by maximum likelihood (Laplace
Approximation) ['bglmerMod']
Family: binomial ( logit )
Formula: correctness ~ task * group + (1 + task | Participant) + (1 +
group | item)
Data: data
Control: glmerControl(optimizer = "bobyqa")
AIC BIC logLik deviance df.resid
3857.8 3957.2 -1913.9 3827.8 5570
Scaled residuals:
Min 1Q Median 3Q Max
-2.0196 -0.3744 -0.2312 -0.1368 6.9534
Random effects:
Groups Name Variance Std.Dev. Corr
item (Intercept) 1.1266 1.0614
groupL2 0.1311 0.3620 -0.12
groupNS 0.2029 0.4504 -0.31 0.17
Participant (Intercept) 0.7582 0.8708
taskpriming 1.2163 1.1029 -0.77
Number of obs: 5585, groups: item, 219; Participant, 46
Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -2.49187 0.28318 -8.800 < 2e-16 ***
taskpriming 1.30911 0.37367 3.503 0.000459 ***
groupL2 -0.04042 0.38322 -0.105 0.916005
groupNS -1.01144 0.36607 -2.763 0.005727 **
taskpriming:groupL2 0.04305 0.48693 0.088 0.929544
taskpriming:groupNS -0.04942 0.46034 -0.107 0.914506
---
Signif. codes: 0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1
Correlation of Fixed Effects:
(Intr) tskprm gropL2 gropNS tsk:L2
taskpriming -0.733
groupL2 -0.660 0.482
groupNS -0.693 0.507 0.509
tskprmng:L2 0.499 -0.632 -0.755 -0.386
tskprmng:NS 0.530 -0.676 -0.390 -0.750 0.508
The model was then subjected to car::Anova for ANOVA type III analysis with
the following output:
car::Anova(paper2analysis1, type = "III")
Analysis of Deviance Table (Type III Wald chisquare tests)
Response: correctness
Chisq Df Pr(>Chisq)
(Intercept) 77.4344 1 < 2.2e-16 ***
task 12.2737 1 0.0004594 ***
group 9.9237 2 0.0070000 **
task:group 0.0391 2 0.9806462
---
Signif. codes: 0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1
I am not sure how to interpret the non-significant interaction in this
case. Does this mean that, although simple effects exist at group level
within one particular task or at task level within one particular group, I
lack sufficient power to conclude those effects are real? If I look at the
simple effects, I do indeed find such effects but am not sure how to
interpret them against the lack of a main interaction. At a practical
level, the interaction, rather than the main effects, is the most important
part of the analysis.
Thank you in advance for any advice.
Francesco
Best,
Frank
[[alternative HTML version deleted]]
_______________________________________________
R-sig-mixed-models at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
-- Inviato da Gmail Mobile
Inviato da Gmail Mobile [[alternative HTML version deleted]]