Hi Francesco,
You already got the unstandardized effect sizes - these simply are the
estimated reaction time differences ("Contrasts Estimate (ms)" in your
table).
If you standardize these differences by dividing by a measure of variation,
e.g. an estimate of the standard deviation of your response variable, you
get so called standardized effect sizes. I guess the latter is what your
reviewer is looking for but (as also mentioned in [1]) one can argue that
this standardization has its own pitfalls.
Best,
Maarten
On Sat, 5 Oct 2019, 21:05 Francesco Romano <fbromano77 at gmail.com> wrote:
Hi Maarten, Thank you so much for your suggestions. In the source that you recommended, a paper is mentioned that advises reporting unstandardised effect sizes but I?m not sure how to do that. Is it the same as the division you explained in your message? If so, how would I obtain the estimated means for the specific contrasts in my table? Francesco On Sat, Oct 5, 2019 at 12:13 PM Maarten Jung < Maarten.Jung at mailbox.tu-dresden.de> wrote:
Dear Francesco, I don't think there is a "standard" way to calculate effect sizes for linear mixed models due to the way the variance is partitioned (see e.g. [1]). One way to compute something similar to Cohen's d would be to divide the difference between the estimated means of two conditions by a rough estimate of the standard deviation of the response variable which you can get by sd(predict(your_model_name)) Best, Maarten [1] https://afex.singmann.science/forums/topic/compute-effect-sizes-for-mixed-objects#post-295 On Sat, Oct 5, 2019 at 10:01 AM Francesco Romano <fbromano77 at gmail.com> wrote:
Dear all,
A journal has asked that I determine the effect sizes for a series of
dummy-coded contrasts from the following ME model:
RT ~ Group * Grammaticality + (1 + Grammaticality | Participant) +
(1 + Group | item)
Here RT is my continuous outcome variable measured in milliseconds,
Group
is a factor with 3 levels (NS, L2, and HL), and Grammaticality a factor with 2 levels (gr and ungr). After relevelling ?NOTE: I am deliberately omitting the call for each new relevelled model here? I obtained a
series
of contrasts which are tabulated below (not sure you can view this
whole):
Reference level Contrasts Estimate (ms) Effect size (Cohen?s *d*) SE df *t* *p* HL GR vs UNGR -213 89 72.13 -2.399 < .05* L2 GR vs UNGR -408 90 74.18 -4.513 < .001*** L1 GR vs UNGR -111 73 70.02 -1.520
.05
HL > L2 -25 191 43.48 -.135
.05
GR L1 > HL 400 175 43.81 2.286 < .05* L1 > L2 374 179 43.59 2.092 < .05* HL > L2 -219 179 42.70 -1.226
.05
UNGR L1 > HL 298 164 43 1.817
.05
L1> L2 77 166 42.03 .469
.05
How would I go about determining the Cohen's *d* for each of the
contrasts?
The model call is: Linear mixed model fit by REML. t-tests use Satterthwaite's method ['lmerModLmerTest'] Formula: RT ~ Group * Grammaticality + (1 + Grammaticality |
Participant) +
(1 + Group | item)
Data: RTanalysis
REML criterion at convergence: 52800
Scaled residuals:
Min 1Q Median 3Q Max
-2.1696 -0.6536 -0.1654 0.5060 5.0134
Random effects:
Groups Name Variance Std.Dev. Corr
item (Intercept) 71442 267.29
GroupL2 1144 33.82 0.80
GroupNS 9951 99.76 -0.43 -0.88
Participant (Intercept) 235216 484.99
Grammaticalityungr 50740 225.25 -0.39
Residual 378074 614.88
Number of obs: 3342, groups: item, 144; Participant, 46
Fixed effects:
Estimate Std. Error df t value Pr(>|t|)
(Intercept) 2801.98 136.70 48.85 20.498 <2e-16
***
GroupL2 -25.86 191.20 43.48 -0.135 0.8931 GroupNS -400.63 175.22 43.81 -2.286 0.0271
*
Grammaticalityungr -213.87 89.17 72.13 -2.399 0.0190
*
GroupL2:Grammaticalityungr -194.57 107.25 42.55 -1.814 0.0767
.
GroupNS:Grammaticalityungr 102.31 99.39 43.45 1.029 0.3090
---
Signif. codes: 0 ?***? 0.001 ?**? 0.01 ?*? 0.05 ?.? 0.1 ? ? 1
Correlation of Fixed Effects:
(Intr) GropL2 GropNS Grmmtc GrL2:G
GroupL2 -0.672
GroupNS -0.744 0.526
Grmmtcltyng -0.404 0.222 0.260
GrpL2:Grmmt 0.259 -0.391 -0.205 -0.589
GrpNS:Grmmt 0.299 -0.202 -0.392 -0.702 0.540
convergence code: 0
Model failed to converge with max|grad| = 0.0477764 (tol = 0.002,
component
1) The distribution of the outcome is fairly normal and the overall mean, without considering the two fixed effects, is very close to the means of each of the three groups (without considering the effect of
Grammaticality)
as well as the means of the two levels of grammaticality (without considering the effect of group). The package simR can simulate data to determine power, amongst other
things,
but I am not sure how to do this for models with interactions such as
mine.
Use of simR is recommended in Brysbaert and Stevens (2018) https://www.journalofcognition.org/articles/10.5334/joc.10/. Perhaps
there
is a simpler way of extracting *d *from the stats I already know?
Any help would be greatly appreciated,
Francesco
[[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
-- Inviato da Gmail Mobile