Question about non-significant interactions

---------- Forwarded message ---------
From: Francesco Romano <fbromano77 at gmail.com>
Date: Tue, Jul 9, 2019 at 11:24 PM
Subject: Re: [R-sig-ME] Question about non-significant interactions
To: Fox, John <jfox at mcmaster.ca>


Dear John,

Thanks for the reply. One of my research entails examining the relationship
between 3 groups of speakers, the 3 levels of the group categorical
variable previously mentioned, and two tasks. One prediction is that one
group will perform better than other groups on one test but not the other.

I fit a maximal model using the bglmr function as shown previously, then
used car::Anova to determine main effects. My understanding from previous
interaction with you precisely here on r-sig-me is that the function works
as a form of shortcut to the traditional way of model-fitting/ reduction
via the function anova() comparing models, eliminating terms one at a time.

I hope this is clearer now and yes, the question is more of a statistical
one than an R one, even though I suspect the mixed-effect aspect of the
regression may be relevant to answering it.

Frank

 Tue, Jul 9, 2019 at 11:03 PM Fox, John <jfox at mcmaster.ca> wrote:

Dear Francesco,

I didn't entirely follow your question and I expect that to answer it, it
would be necessary to know more about what your research entails. As you
imply, this seems to be more a statistics question than an R question.

It's

also not clear to me what function you used to fit the mixed-effects
logistic regression.

But I did notice that you're apparently using Anova() for type-III tests
with the default contr.treatment() coding for factors. The main-effect
tests that result are not sensible. As it says in ?Anova:

"Warning
Be careful of type-III tests: For a traditional multifactor ANOVA model
with interactions, for example, these tests will normally only be

sensible

when using contrasts that, for different terms, are orthogonal in the
row-basis of the model, such as those produced by contr.sum, contr.poly,

or

contr.helmert, but not by the default contr.treatment. In a model that
contains factors, numeric covariates, and interactions, main-effect tests
for factors will be for differences over the origin. In contrast (pun
intended), type-II tests are invariant with respect to (full-rank)

contrast

coding. If you don't understand this issue, then you probably shouldn't

use

Anova for type-III tests."

I hope that this is of some help,
 John
-----------------------------
John Fox, Professor Emeritus
McMaster University
Hamilton, Ontario
Canada L8S 4M4
web: socserv.mcmaster.ca/jfox

item

and task by subject.

I am interested in the effect of belonging any of three groups, the

levels

of the group IV, in order to test some a priori predictions. The bayesian
wrapper is to help the model converge.

Here is the output:

summary(paper2analysis1)

Cov prior  : item ~ wishart(df = 5.5, scale = Inf, posterior.scale = cov,
common.scale = TRUE)
           : Participant ~ wishart(df = 4.5, scale = Inf,

posterior.scale =

cov, common.scale = TRUE)
Prior dev  : 6.9466

Generalized linear mixed model fit by maximum likelihood (Laplace
Approximation) ['bglmerMod']
 Family: binomial  ( logit )
Formula: correctness ~ task * group + (1 + task | Participant) + (1 +
 group | item)
   Data: data
Control: glmerControl(optimizer = "bobyqa")

     AIC      BIC   logLik deviance df.resid
  3857.8   3957.2  -1913.9   3827.8     5570

Scaled residuals:
    Min      1Q  Median      3Q     Max
-2.0196 -0.3744 -0.2312 -0.1368  6.9534

Random effects:
 Groups      Name        Variance Std.Dev. Corr
 item        (Intercept) 1.1266   1.0614
             groupL2     0.1311   0.3620   -0.12
             groupNS     0.2029   0.4504   -0.31  0.17
 Participant (Intercept) 0.7582   0.8708
             taskpriming 1.2163   1.1029   -0.77
Number of obs: 5585, groups:  item, 219; Participant, 46

Fixed effects:
                    Estimate Std. Error z value Pr(>|z|)
(Intercept)         -2.49187    0.28318  -8.800  < 2e-16 ***
taskpriming          1.30911    0.37367   3.503 0.000459 ***
groupL2             -0.04042    0.38322  -0.105 0.916005
groupNS             -1.01144    0.36607  -2.763 0.005727 **
taskpriming:groupL2  0.04305    0.48693   0.088 0.929544
taskpriming:groupNS -0.04942    0.46034  -0.107 0.914506
---
Signif. codes:  0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1

Correlation of Fixed Effects:
            (Intr) tskprm gropL2 gropNS tsk:L2
taskpriming -0.733
groupL2     -0.660  0.482
groupNS     -0.693  0.507  0.509
tskprmng:L2  0.499 -0.632 -0.755 -0.386
tskprmng:NS  0.530 -0.676 -0.390 -0.750  0.508

The model was then subjected to car::Anova for ANOVA type III analysis

with

the following output:

car::Anova(paper2analysis1, type = "III")

Analysis of Deviance Table (Type III Wald chisquare tests)

Response: correctness
              Chisq Df Pr(>Chisq)
(Intercept) 77.4344  1  < 2.2e-16 ***
task        12.2737  1  0.0004594 ***
group        9.9237  2  0.0070000 **
task:group   0.0391  2  0.9806462
---
Signif. codes:  0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1

I am not sure how to interpret the non-significant interaction in this
case. Does this mean that, although simple effects exist at group level
within one particular task or at task level within one particular group,

I

lack sufficient power to conclude those effects are real? If I look at

the

simple effects, I do indeed find such effects but am not sure how to
interpret them against the lack of a main interaction. At a practical
level, the interaction, rather than the main effects, is the most

important

part of the analysis.

Thank you in advance for any advice.

Francesco





Best,

Frank

        [[alternative HTML version deleted]]

--
Inviato da Gmail Mobile
--
Inviato da Gmail Mobile

        [[alternative HTML version deleted]]

Dear Francesco,

To answer your question, you should convert your hypothesis in a set of
linear contrasts and test those.

Best regards,

ir. Thierry Onkelinx
Statisticus / Statistician

Vlaamse Overheid / Government of Flanders
INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE AND
FOREST
Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
thierry.onkelinx at inbo.be
Havenlaan 88 bus 73, 1000 Brussel
www.inbo.be

///////////////////////////////////////////////////////////////////////////////////////////
To call in the statistician after the experiment is done may be no more
than asking him to perform a post-mortem examination: he may be able to say
what the experiment died of. ~ Sir Ronald Aylmer Fisher
The plural of anecdote is not data. ~ Roger Brinner
The combination of some data and an aching desire for an answer does not
ensure that a reasonable answer can be extracted from a given body of data.
~ John Tukey
///////////////////////////////////////////////////////////////////////////////////////////

<https://www.inbo.be>


Op di 9 jul. 2019 om 23:25 schreef Francesco Romano <fbromano77 at gmail.com>:

---------- Forwarded message ---------
From: Francesco Romano <fbromano77 at gmail.com>
Date: Tue, Jul 9, 2019 at 11:24 PM
Subject: Re: [R-sig-ME] Question about non-significant interactions
To: Fox, John <jfox at mcmaster.ca>


Dear John,

Thanks for the reply. One of my research entails examining the relationship
between 3 groups of speakers, the 3 levels of the group categorical
variable previously mentioned, and two tasks. One prediction is that one
group will perform better than other groups on one test but not the other.

I fit a maximal model using the bglmr function as shown previously, then
used car::Anova to determine main effects. My understanding from previous
interaction with you precisely here on r-sig-me is that the function works
as a form of shortcut to the traditional way of model-fitting/ reduction
via the function anova() comparing models, eliminating terms one at a time.

I hope this is clearer now and yes, the question is more of a statistical
one than an R one, even though I suspect the mixed-effect aspect of the
regression may be relevant to answering it.

Frank

Tue, Jul 9, 2019 at 11:03 PM Fox, John <jfox at mcmaster.ca> wrote:

Dear Francesco,

I didn't entirely follow your question and I expect that to answer it, it
would be necessary to know more about what your research entails. As you
imply, this seems to be more a statistics question than an R question.
It's
also not clear to me what function you used to fit the mixed-effects
logistic regression.

But I did notice that you're apparently using Anova() for type-III tests
with the default contr.treatment() coding for factors. The main-effect
tests that result are not sensible. As it says in ?Anova:

"Warning
Be careful of type-III tests: For a traditional multifactor ANOVA model
with interactions, for example, these tests will normally only be
sensible
when using contrasts that, for different terms, are orthogonal in the
row-basis of the model, such as those produced by contr.sum, contr.poly,
or
contr.helmert, but not by the default contr.treatment. In a model that
contains factors, numeric covariates, and interactions, main-effect tests
for factors will be for differences over the origin. In contrast (pun
intended), type-II tests are invariant with respect to (full-rank)
contrast
coding. If you don't understand this issue, then you probably shouldn't
use
Anova for type-III tests."

I hope that this is of some help,
John
-----------------------------
John Fox, Professor Emeritus
McMaster University
Hamilton, Ontario
Canada L8S 4M4
web: socserv.mcmaster.ca/jfox