subjects within groups and effects of group

However, I would like to know how effects of spectral power (Alpha)
over Drive vary as a function of Condition.

I may have lost track of your experimental design by this point (I have
experiments of my own to keep track of :) ), but my point had more to do
with the symmetry of interactions -- it may be that the impact of
Condition is modulated by endogenous fluctuations in alpha power. Or
given that alpha power is related to certain aspects of attention, it
may be that alpha power interacts with / modulates the effect of
Condition simply because it serves as a decent proxy for aspects of
attention which are relevant. The same holds for a potential interaction
with Drive and especially with SeqNo (as you could expect attention to
vary somewhat over the course of an experiment).

In other words, because alpha correlates to some extent with attention
and level of attention of course impacts performance on cognitive tasks
(including operating complex machinery such as a car), I would expect to
find some interaction effects between alpha power and other aspects of
the experimental manipulations. This is more a experimental psychology
consideration and less-so a statistical one. On a related statistical
note, I wouldn't include any interactions with alpha power in the random
effects as I suspect that would unnecessarily overparameterize the model.

So I would probably start with a model like:

log(RT) ~ 1 + (PzAlpha + Condition + Drive + SeqNo)^3 + (1 + PzAlpha |
subject) + (1 | trial)

This includes main effects and two- and three-way interactions but
excludes four-way interactions, which are hard to interpret, hard to
compute and probably won't drastically improve your model fit anyway
(based on my experience with these types of data). I allow the effect of
alpha power to vary by subject, in case there are baseline differences
in alpha power that affect the correlation with attention, but I would
drop this effect immediately if the model doesn't converge. Condition is
between subjects, so allowing it to vary by subject doesn't make much
sense. Trial only gets a random intercept because it is just capturing
some notion of fixed repetition and I wouldn't expect the other effects
of the experimental manipulation to vary strongly by Trial.

I would then check the model fit (e.g. by plotting predictions vs.
observed data). If the fit is good enough, then I would try to make some
inferences based on it, even knowing the model isn't perfect -- after
all, "all models are wrong, but some models are useful". You could try
to add the four-way interaction back in or by-subject slopes for SeqNo,
etc. but I suspect you'll have trouble getting those models to converge
with only 1839 observations and they probably won't fit the data that
much better (based on my experience with this type of data).

For lmer() and car::Anova(), it doesn't really matter if your predictors
are between or within subjects / items /etc. Between-subjects
manipulations tend provide better estimates (Andrew Gelman frequently
brings this up on his blog), which in practical terms means that they
have better power, but that's about it.

For post-hoc tests, I would recommend the emmeans package (successor to
lsmeans). The documentation is rather extensive, including lots of notes
and examples on interactions.

This is slowly drifting away from statistical issues and more towards
neuro/psych issues -- if other people on the list feel like it's gone
too far away from mixed models, just let us know. :)

Best,
Phillip


On 26/01/18 23:37, P Greenwood wrote:

Dear Dr. Alday

Could you elaborate a bit on your answer to my question in decomposing
the effect of condition. Condition was randomly assigned to two groups
of participants. I did include both levels of Condition in my analysis
(the output I sent originally). One approach might be to use the Anova
command to perform a likelihood ratio test to compare a model that
includes condition with a model that does not include condition.
(Perhaps that is what you mean by ?test the difference??)  However, I
would like to know how effects of spectral power (Alpha) over Drive vary
as a function of Condition. 

Thanks also for the information on sequence information. 

Thanks so much

Pam


P.M. Greenwood, Ph.D.
/Associate Professor of Psychology/
/Editorial Board, NeuroImage/
/David King Hall 2052/
/George Mason University/
/MSN 3F5, 4400 University Drive/
/Fairfax, VA 22030-4444/

/Ph: /703 993-4268
/fax/: 703 993-1359
/email: Pgreenw1 at gmu.edu <mailto:Pgreenw1 at gmu.edu>/
/http://psychology.gmu.edu/people/pgreenw1/

On Jan 25, 2018, at 5:44 PM, Alday, Phillip <Phillip.Alday at mpi.nl
<mailto:Phillip.Alday at mpi.nl>> wrote:

Completely agree with Thierry here.

In addition to the usual considerations about the bias-variance
tradeoff and partial pooling, you need to have things in one model if
you really want to compare them. The Difference Between ?Significant?
and ?Not Significant? is not Itself Statistically Significant (Gelman
and Stern 2012, doi:10.1198/000313006X152649
<https://secure-web.cisco.com/1K0nLxMcHeaEQ3zltL5zGKV6Wf3PuASMvvhRXl8z8qKf_35cqaaAeKSo2Q9Jln-azfrr34PbzOOlzZaZYuDWxL5arm_4R9mtGR3Sfsj9-ShTlNCxMGN06gFN5920BieQ1AbiUfjBLRNCvTRcAUTTNlwGGWzVpfqdGudEkiQ-lN89uB95el2DuEfyJW_E5dtTTKpWwSEYaJQc-1ZqKU74d4imV2ENHCLwror_8EZuYaZ51caefF2SHC0JYTTA_uKYgP_FECA8Q_-j4IjYDn_ZmjdvhekseOw9ixb4f2uGavmMS9iVdBAbeqT7xIO8l66l4yZaRUC7isfxVvlhJDIVb4-gRWFgjzG6uSmMzHBhJL44Y26l0SXjQcNYytqFIhCBmM8ZP-SmusBWVAQAymVxHbPAYWXgqpQif7ESbchFfnl3NrBmMQzazxgNQt-ymStEY0d-GMFAIpZfZxFmXh9Y570HQn5LungV8fQy2VbMa-yw/https%3A%2F%2Fdoi.org%2F10.1198%2F000313006X152649>),
so if you care about the significance of the difference, then you need
to actually test the difference!

For your other question

Trial refers to stimulus events.  The stimuli are the same on each
Trial, although the order of the Trials varies between Drives.   But,
yes, Trial is a sequence number for the repetition so that there
could be some adaptation or change in response related to number of
exposures.  (Assuming that is what you meant).  How would I include
Trial as a continuous fixed effect?

I would use slightly different names to make things clear. Separate
'Trial' (a fixed series of stimuli) from SeqNo (the sequential
position of a given Trial within a Drive).

Then your model looks something like this:

lmer(RT ~ 1 + Condition*PzAlpha + Drive + SeqNo + (1 | subject) + (1 |
trial)

I've left out any interactions there, but I suspect you'll at least
have an interaction with alpha and sequence number -- I imagine that
later trials (i.e. higher sequence numbers) will have worse RTs
(exhaustion effects) as will trials with higher alpha power and that
this two effects will enhance each other.

Including sequence information in the model has received some
attention in the psycholinguistic as well as the broader psychology
literature as a way of controlling for adapt ion effects. GAMMs have
been proposed for such cases to allow for non linear adaptation
effects, but I wouldn't mess around with that until you feel much more
comfortable with the standard LMMs.

And of course, if SeqNo doesn't improve model fit, you can simply omit
it for parsimony and easy of both interpretation and fitting.

Phillip


On 25/01/18 17:33, Thierry Onkelinx wrote:

Dear Pam,

I'd probably combine both datasets in a single analysis.

Best regards,

ir. Thierry Onkelinx
Statisticus / Statistician

Vlaamse Overheid / Government of Flanders
INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE
AND FOREST
Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
thierry.onkelinx at inbo.be
Havenlaan 88 bus 73, 1000 Brussel
www.inbo.be

///////////////////////////////////////////////////////////////////////////////////////////
To call in the statistician after the experiment is done may be no
more than asking him to perform a post-mortem examination: he may be
able to say what the experiment died of. ~ Sir Ronald Aylmer Fisher
The plural of anecdote is not data. ~ Roger Brinner
The combination of some data and an aching desire for an answer does
not ensure that a reasonable answer can be extracted from a given body
of data. ~ John Tukey
///////////////////////////////////////////////////////////////////////////////////////////




2018-01-24 14:02 GMT+01:00 P Greenwood <pgreenw1 at gmu.edu>:

Dear Drs Alday and Onkelinx

I wondered if you had thoughts on the best way to conduct followup analysis
of the between-subjects Condition to which people were randomly assigned.

Pam Greenwood

P.M. Greenwood, Ph.D.
Associate Professor of Psychology
Editorial Board, NeuroImage
David King Hall 2052
George Mason University
MSN 3F5, 4400 University Drive
Fairfax, VA 22030-4444

Ph: 703 993-4268
fax: 703 993-1359
email: Pgreenw1 at gmu.edu
http://psychology.gmu.edu/people/pgreenw1

On Jan 19, 2018, at 8:09 AM, P Greenwood <pgreenw1 at gmu.edu> wrote:

Thanks to you both.

Trial refers to stimulus events.  The stimuli are the same on each Trial,
although the order of the Trials varies between Drives.   But, yes, Trial is
a sequence number for the repetition so that there could be some adaptation
or change in response related to number of exposures.  (Assuming that is
what you meant).  How would I include Trial as a continuous fixed effect?

If the effect of Condition were ?significant.? how would one decompose that
to examine each group (Condition) separately?

Regards

Pam


P.M. Greenwood, Ph.D.
Associate Professor of Psychology
Editorial Board, NeuroImage
David King Hall 2052
George Mason University
MSN 3F5, 4400 University Drive
Fairfax, VA 22030-4444

Ph: 703 993-4268
fax: 703 993-1359
email: Pgreenw1 at gmu.edu
http://psychology.gmu.edu/people/pgreenw1

On Jan 19, 2018, at 5:58 AM, Phillip Alday <phillip.alday at mpi.nl> wrote:

Dear Pam, (dear Thierry,)

if I'm reading the description correctly, Pam is conceiving of Trial as
being an "Item"-type factor (crossed with subject). To rephrase
Thierry's comment a bit -- if Trial corresponds to an Item (concrete
stimulus realization sampled from the population of possible stimuli for
this manipulation) that is the same across subjects, then this is a good
way to model that. If Trial doesn't correspond to an invariant set of
items, but is rather just repetitions of the same task (perhaps with
some random variation that isn't identical across subjects), then
modeling Trial as a random effect doesn't really help much. However, if
Trial is just a sequence number for the repetition, it might make sense
to instead include Trial as a continuous fixed effect in order to model
adaptation effects.

Best,
Phillip

On 19/01/18 10:44, Thierry Onkelinx wrote:

Dear Pam,

You are handling condition and subject correctly.

There might be a problem with trial. Does trial indicates dependent
replication of the study? Is there a common effect of trial X for all
subjects? Because that is what your current model assumes. In case the
trials are independent, then you don't need to include it in the
model.

Note that Condition + PzAlpha + PzAlpha*Condition is verbose. You can
write it as PzAlpha*Condition.

Best regards,

ir. Thierry Onkelinx
Statisticus / Statistician

Vlaamse Overheid / Government of Flanders
INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE
AND FOREST
Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
thierry.onkelinx at inbo.be
Havenlaan 88 bus 73, 1000 Brussel
www.inbo.be

///////////////////////////////////////////////////////////////////////////////////////////
To call in the statistician after the experiment is done may be no
more than asking him to perform a post-mortem examination: he may be
able to say what the experiment died of. ~ Sir Ronald Aylmer Fisher
The plural of anecdote is not data. ~ Roger Brinner
The combination of some data and an aching desire for an answer does
not ensure that a reasonable answer can be extracted from a given body
of data. ~ John Tukey
///////////////////////////////////////////////////////////////////////////////////////////




2018-01-18 18:51 GMT+01:00 P Greenwood <pgreenw1 at gmu.edu>:

Hello

I wanted some advice about handling subjects within groups and effects of
group (randomly assigned).  I want to predict reaction time (RT) as a
function of  ?Condition,?  alpha band power (PzAlpha), and drive. People
(subjects) are randomly assigned to Condition, of which there are two. Each
person has data from 5 drives, and for each drive there are 10 trials.
There are 19 subjects in one group and 20 in the other.

My question is this: Am I handling the ?between subjects? factor of
Condition correctly?  Also, am I treating subjects within group correctly?
I am pasting in some of my data.  The output is below.

Regards

Pam Greenwood

library(lme4)
library(lmerTest)
INFAST_Behavioral <- read.csv(??.
na.omit(INFAST_Behavioral)
INFAST_Behavioral$RT = scale(INFAST_Behavioral$RT, center = TRUE, scale =
TRUE)
INFAST_Behavioral$PzAlpha = scale(INFAST_Behavioral$PzAlpha, center = TRUE,
scale = TRUE)
sumModelInteraction <- lmer(RT ~ 1 + (Condition + PzAlpha + Drive +
PzAlpha*Condition) + (1 | subject) + (1 | trial), data = INFAST_Behavioral)
summary(sumModelInteraction)

subject Condition               Drive           trial   FzAlpha CzAlpha
PzAlpha FzTheta CzTheta PzTheta FzDelta CzDelta PzDelta         RT      ACC
1       HumanLanguage   1       1       -1.41   -4.3585 -5.5431 6.1516
1.5911  3.6247  22.38   18.181 13.812          1568.984857     1
1       HumanLanguage   1       2       -7.8605 2.0156  4.7392  15.992
12.122  6.9088  26.861 20.592  16.326  1721.359714     1
1       HumanLanguage   1       3       -2.6982 -5.6067 -10.038 6.285
5.5172  1.2894  13.565 12.981  11.63   1257.092571     1
1       HumanLanguage   1       4       3.3975  4.8789  -1.3249 7.0177
9.6703  6.1539  10.231 12.261  12.485  1559.461429     1
?(skipping to Subject 2)
2       HumanLanguage                   1       1       1.6791  2.8887
0.28174 -11.387 -9.9352 3.5936 -1.5767 3.9401  6.7201          1302.328857
1
2       HumanLanguage   1       2       -13.284 -8.2603 -6.6124 -5.9373
-8.7551 0.10394 4.5621 10.204  12.261  969.0088571     1
2       HumanLanguage   1       3       -0.048973       1.1329  0.67399
-2.1432 2.5077  -2.4641 9.4667 10.883  7.1396  721.3997143     1
2       HumanLanguage   1       4       5.0779  6.8916  6.3892  -1.8682
3.1637  7.9712  8.0994 10.883  10.975  707.1145714     1
2       HumanLanguage   1       5       -7.0495 -2.782  3.1668  8.4332
10.646  9.3726  -3.5937 -7.3769 5.4472  892.8214286     1
2       HumanLanguage   1       6       -1.462  -8.1223 -6.5896 -10.895
-5.6311 0.39941 7.5473 12.783  14.698  611.8802857     1
2       HumanLanguage   1       7       -2.6402 -5.1213 -3.7372 3.4542
4.2234  -0.99898        1.4089 4.1976  0.56587 761.8742857     1
2       HumanLanguage   1       8       3.4393  4.6302  1.5525  1.4604
3.1716  3.1622  -2.3427 2.908 4.2259  680.9251429     1
2       HumanLanguage   1       9       -0.81024        -0.21642
-2.3876 2.5839  4.7307  1.5441 3.3761  8.4485  12.02   769.0168571     1
2       HumanLanguage   1       10      -6.4045 -4.4937 -2.2449 0.94456
2.7048  0.65565 -1.9791 0.26436 1.8435  885.6788571     1

Results:

Linear mixed model fit by REML t-tests use Satterthwaite approximations to
degrees of freedom [
lmerMod]
Formula: RT ~ 1 + (Condition + PzAlpha + Drive + PzAlpha * Condition) +
   (1 | subject) + (1 | trial)
  Data: INFAST_Behavioral

REML criterion at convergence: 3876.4

Scaled residuals:
   Min      1Q  Median      3Q     Max
-3.4308 -0.5227 -0.1194  0.3547  8.4095

Random effects:
Groups   Name        Variance Std.Dev.
subject  (Intercept) 0.580073 0.76163
trial    (Intercept) 0.004778 0.06912
Residual             0.434918 0.65948
Number of obs: 1839, groups:  subject, 39; trial, 10

Fixed effects:
                                        Estimate               Std. Error
df t value Pr(>|t|)
(Intercept)                        -0.27054    0.17607   40.80000  -1.537
0.13213
ConditionMachineLang 0.41644    0.24595   36.90000   1.693  0.09884 .
PzAlpha                             0.01192    0.02411 1797.40000   0.494
0.62117
Drive                               0.02948    0.01083 1788.40000   2.722
0.00655 **
ConditionMachineLanguage:PzAlpha   -0.01998    0.03476 1803.10000  -0.575
0.56560
---
Signif. codes:  0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1
P.M. Greenwood, Ph.D.
Associate Professor of Psychology
Editorial Board, NeuroImage
David King Hall 2052
George Mason University
MSN 3F5, 4400 University Drive
Fairfax, VA 22030-4444

Ph: 703 993-4268
fax: 703 993-1359
email: Pgreenw1 at gmu.edu
http://psychology.gmu.edu/people/pgreenw1


       [[alternative HTML version deleted]]