Question about non-significant interactions
I am currently out of the office until July 5th. I will respond to your email upon my return.
On Jul 10, 2019, at 12:51 AM, Thierry Onkelinx via R-sig-mixed-models <r-sig-mixed-models at r-project.org> wrote:
Dear Francesco, To answer your question, you should convert your hypothesis in a set of linear contrasts and test those. Best regards, ir. Thierry Onkelinx Statisticus / Statistician Vlaamse Overheid / Government of Flanders INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE AND FOREST Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance thierry.onkelinx at inbo.be Havenlaan 88 bus 73, 1000 Brussel www.inbo.be /////////////////////////////////////////////////////////////////////////////////////////// To call in the statistician after the experiment is done may be no more than asking him to perform a post-mortem examination: he may be able to say what the experiment died of. ~ Sir Ronald Aylmer Fisher The plural of anecdote is not data. ~ Roger Brinner The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data. ~ John Tukey /////////////////////////////////////////////////////////////////////////////////////////// <https://www.inbo.be> Op di 9 jul. 2019 om 23:25 schreef Francesco Romano <fbromano77 at gmail.com>: ---------- Forwarded message --------- From: Francesco Romano <fbromano77 at gmail.com> Date: Tue, Jul 9, 2019 at 11:24 PM Subject: Re: [R-sig-ME] Question about non-significant interactions To: Fox, John <jfox at mcmaster.ca> Dear John, Thanks for the reply. One of my research entails examining the relationship between 3 groups of speakers, the 3 levels of the group categorical variable previously mentioned, and two tasks. One prediction is that one group will perform better than other groups on one test but not the other. I fit a maximal model using the bglmr function as shown previously, then used car::Anova to determine main effects. My understanding from previous interaction with you precisely here on r-sig-me is that the function works as a form of shortcut to the traditional way of model-fitting/ reduction via the function anova() comparing models, eliminating terms one at a time. I hope this is clearer now and yes, the question is more of a statistical one than an R one, even though I suspect the mixed-effect aspect of the regression may be relevant to answering it. Frank Tue, Jul 9, 2019 at 11:03 PM Fox, John <jfox at mcmaster.ca> wrote: Dear Francesco, I didn't entirely follow your question and I expect that to answer it, it would be necessary to know more about what your research entails. As you imply, this seems to be more a statistics question than an R question. It's also not clear to me what function you used to fit the mixed-effects logistic regression. But I did notice that you're apparently using Anova() for type-III tests with the default contr.treatment() coding for factors. The main-effect tests that result are not sensible. As it says in ?Anova: "Warning Be careful of type-III tests: For a traditional multifactor ANOVA model with interactions, for example, these tests will normally only be sensible when using contrasts that, for different terms, are orthogonal in the row-basis of the model, such as those produced by contr.sum, contr.poly, or contr.helmert, but not by the default contr.treatment. In a model that contains factors, numeric covariates, and interactions, main-effect tests for factors will be for differences over the origin. In contrast (pun intended), type-II tests are invariant with respect to (full-rank) contrast coding. If you don't understand this issue, then you probably shouldn't use Anova for type-III tests." I hope that this is of some help, John ----------------------------- John Fox, Professor Emeritus McMaster University Hamilton, Ontario Canada L8S 4M4 web: socserv.mcmaster.ca/jfox
________________________________________ From: R-sig-mixed-models [r-sig-mixed-models-bounces at r-project.org] on behalf of Francesco Romano [fbromano77 at gmail.com] Sent: July 9, 2019 9:49 AM To: r-sig-mixed-models at r-project.org Subject: [R-sig-ME] Question about non-significant interactions Dear all, I have more of a theoretical than practical question for you. The model I am using has two IVs, group (3 levels) and task (2 levels), and a categorical DV (correct versus incorrect), hence logistic regression. Random effects for subjects and items, as well as slopes for group by item and task by subject. I am interested in the effect of belonging any of three groups, the levels of the group IV, in order to test some a priori predictions. The bayesian wrapper is to help the model converge. Here is the output: summary(paper2analysis1) Cov prior ?: item ~ wishart(df = 5.5, scale = Inf, posterior.scale = cov, common.scale = TRUE) ??????????: Participant ~ wishart(df = 4.5, scale = Inf, posterior.scale = cov, common.scale = TRUE) Prior dev ?: 6.9466 Generalized linear mixed model fit by maximum likelihood (Laplace Approximation) ['bglmerMod'] Family: binomial ?( logit ) Formula: correctness ~ task * group + (1 + task | Participant) + (1 + group | item) ??Data: data Control: glmerControl(optimizer = "bobyqa") ????AIC ?????BIC ??logLik deviance df.resid ?3857.8 ??3957.2 ?-1913.9 ??3827.8 ????5570 Scaled residuals: ???Min ?????1Q ?Median ?????3Q ????Max -2.0196 -0.3744 -0.2312 -0.1368 ?6.9534 Random effects: Groups ?????Name ???????Variance Std.Dev. Corr item ???????(Intercept) 1.1266 ??1.0614 ????????????groupL2 ????0.1311 ??0.3620 ??-0.12 ????????????groupNS ????0.2029 ??0.4504 ??-0.31 ?0.17 Participant (Intercept) 0.7582 ??0.8708 ????????????taskpriming 1.2163 ??1.1029 ??-0.77 Number of obs: 5585, groups: ?item, 219; Participant, 46 Fixed effects: ???????????????????Estimate Std. Error z value Pr(>|z|) (Intercept) ????????-2.49187 ???0.28318 ?-8.800 ?< 2e-16 *** taskpriming ?????????1.30911 ???0.37367 ??3.503 0.000459 *** groupL2 ????????????-0.04042 ???0.38322 ?-0.105 0.916005 groupNS ????????????-1.01144 ???0.36607 ?-2.763 0.005727 ** taskpriming:groupL2 ?0.04305 ???0.48693 ??0.088 0.929544 taskpriming:groupNS -0.04942 ???0.46034 ?-0.107 0.914506 --- Signif. codes: ?0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1 Correlation of Fixed Effects: ???????????(Intr) tskprm gropL2 gropNS tsk:L2 taskpriming -0.733 groupL2 ????-0.660 ?0.482 groupNS ????-0.693 ?0.507 ?0.509 tskprmng:L2 ?0.499 -0.632 -0.755 -0.386 tskprmng:NS ?0.530 -0.676 -0.390 -0.750 ?0.508 The model was then subjected to car::Anova for ANOVA type III analysis with the following output: car::Anova(paper2analysis1, type = "III") Analysis of Deviance Table (Type III Wald chisquare tests) Response: correctness ?????????????Chisq Df Pr(>Chisq) (Intercept) 77.4344 ?1 ?< 2.2e-16 *** task ???????12.2737 ?1 ?0.0004594 *** group ???????9.9237 ?2 ?0.0070000 ** task:group ??0.0391 ?2 ?0.9806462 --- Signif. codes: ?0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1 I am not sure how to interpret the non-significant interaction in this case. Does this mean that, although simple effects exist at group level within one particular task or at task level within one particular group, I lack sufficient power to conclude those effects are real? If I look at the simple effects, I do indeed find such effects but am not sure how to interpret them against the lack of a main interaction. At a practical level, the interaction, rather than the main effects, is the most important part of the analysis. Thank you in advance for any advice. Francesco Best, Frank ???????[[alternative HTML version deleted]] _______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models -- Inviato da Gmail Mobile -- Inviato da Gmail Mobile ???????[[alternative HTML version deleted]] _______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models [[alternative HTML version deleted]] _______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
[[alternative text/enriched version deleted]]