likelihood-ratio tests in conflict with coefficiants in maximal random effect model
On Mar 4, 2014, at 3:10 AM, Emilia Ellsiepen <emilia.ellsiepen at gmail.com> wrote:
Thank you for your support, Roger - actually the problem vanished when I used sum-coding (after installing the packages and reloading the data, I accidentally skipped this step yesterday). So, in fact with the old lme4.0 version, the model did converge. I pasted in all model summaries and LRTs for dummy and sum coding below. Would you recommend to always use the old version then?
Thanks so much, Emilia, for following up. This is very interesting. Something doesn?t quite add up, though, in the model summaries and LRTs that you pasted below, because the log-likelihoods in the model summaries are not the same as the log-likelihoods reported in the results of anova(). I notice that you?re not using REML=F as an argument in your call to lmer(). Could you perhaps rerun all four model-fitting functions and the two anova() calls with REML=F, and show us the complete results? Best & thanks again, Roger
*** with dummy coding:
m1 = lmer(score ~ Order*Voice + (Order*Voice|subject)+(Order*Voice|sentence) ,rawdata)
Warning message: In mer_finalize(ans) : singular convergence (7)
summary(m1)
Linear mixed model fit by REML
Formula: score ~ Order * Voice + (Order * Voice | subject) + (Order *
Voice | sentence)
Data: rawdata
AIC BIC logLik deviance REMLdev
1626 1740 -787.9 1562 1576
Random effects:
Groups Name Variance Std.Dev. Corr
subject (Intercept) 0.097201 0.31177
OrderDat second 0.055057 0.23464 -0.690
VoicePassive 0.161333 0.40166 -0.952 0.878
OrderDat second:VoicePassive 0.182879 0.42764 0.732 -0.986 -0.900
sentence (Intercept) 0.015671 0.12518
OrderDat second 0.068464 0.26166 -0.663
VoicePassive 0.025271 0.15897 -0.237 -0.031
OrderDat second:VoicePassive 0.053708 0.23175 0.031 -0.589 -0.425
Residual 0.465541 0.68231
Number of obs: 720, groups: subject, 36; sentence, 20
Fixed effects:
Estimate Std. Error t value
(Intercept) 0.16341 0.07791 2.097
OrderDat second -0.01835 0.10062 -0.182
VoicePassive -0.05604 0.10449 -0.536
OrderDat second:VoicePassive 0.54250 0.13458 4.031
Correlation of Fixed Effects:
(Intr) OrdrDs VcPssv
OrderDtscnd -0.647
VoicePassiv -0.754 0.459
OrdrDscn:VP 0.509 -0.717 -0.729
mi = lmer(score ~ Order+Voice + (Order*Voice|subject)+(Order*Voice|sentence) ,rawdata) summary(mi)
Linear mixed model fit by REML
Formula: score ~ Order + Voice + (Order * Voice | subject) + (Order *
Voice | sentence)
Data: rawdata
AIC BIC logLik deviance REMLdev
1634 1744 -793.2 1574 1586
Random effects:
Groups Name Variance Std.Dev. Corr
subject (Intercept) 0.115077 0.33923
OrderDat second 0.094406 0.30726 -0.738
VoicePassive 0.211833 0.46025 -0.951 0.893
OrderDat second:VoicePassive 0.330228 0.57465 0.761 -0.999 -0.902
sentence (Intercept) 0.019567 0.13988
OrderDat second 0.111327 0.33366 -0.814
VoicePassive 0.048545 0.22033 -0.451 0.420
OrderDat second:VoicePassive 0.184456 0.42948 0.478 -0.771 -0.793
Residual 0.463180 0.68057
Number of obs: 720, groups: subject, 36; sentence, 20
Fixed effects:
Estimate Std. Error t value
(Intercept) 0.009676 0.066665 0.145
OrderDat second 0.276029 0.069910 3.948
VoicePassive 0.237470 0.071133 3.338
Correlation of Fixed Effects:
(Intr) OrdrDs
OrderDtscnd -0.472
VoicePassiv -0.631 -0.160
anova(m1,mi)
Data: rawdata Models: mi: score ~ Order + Voice + (Order * Voice | subject) + (Order * mi: Voice | sentence) m1: score ~ Order * Voice + (Order * Voice | subject) + (Order * m1: Voice | sentence) Df AIC BIC logLik Chisq Chi Df Pr(>Chisq) mi 24 1622.5 1732.4 -787.24 m1 25 1611.7 1726.2 -780.86 12.772 1 0.0003518 *** --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 *** with sum-coding
contrasts(rawdata$Order) = contr.sum(2) contrasts(rawdata$Voice) = contr.sum(2) m1 = lmer(score ~ Order*Voice + (Order*Voice|subject)+(Order*Voice|sentence) ,rawdata) summary(m1)
Linear mixed model fit by REML
Formula: score ~ Order * Voice + (Order * Voice | subject) + (Order *
Voice | sentence)
Data: rawdata
AIC BIC logLik deviance REMLdev
1631 1746 -790.7 1562 1581
Random effects:
Groups Name Variance Std.Dev. Corr
subject (Intercept) 1.9949e-02 0.1412424
Order1 1.4195e-05 0.0037676 -1.000
Voice1 1.3413e-02 0.1158136 0.833 -0.833
Order1:Voice1 1.1566e-02 0.1075434 0.286 -0.286 0.615
sentence (Intercept) 2.9850e-03 0.0546352
Order1 1.1405e-02 0.1067934 0.257
Voice1 5.8322e-03 0.0763687 0.297 -0.438
Order1:Voice1 3.4893e-03 0.0590699 -0.884 0.182 -0.328
Residual 4.6543e-01 0.6822234
Number of obs: 720, groups: subject, 36; sentence, 20
Fixed effects:
Estimate Std. Error t value
(Intercept) 0.26184 0.03674 7.127
Order1 -0.12645 0.03489 -3.625
Voice1 -0.10761 0.03620 -2.972
Order1:Voice1 0.13562 0.03380 4.013
Correlation of Fixed Effects:
(Intr) Order1 Voice1
Order1 0.047
Voice1 0.331 -0.150
Order1:Voc1 -0.018 0.046 0.113
mi = lmer(score ~ Order+Voice + (Order*Voice|subject)+(Order*Voice|sentence) ,rawdata) summary(mi)
Linear mixed model fit by REML
Formula: score ~ Order + Voice + (Order * Voice | subject) + (Order *
Voice | sentence)
Data: rawdata
AIC BIC logLik deviance REMLdev
1637 1747 -794.6 1575 1589
Random effects:
Groups Name Variance Std.Dev. Corr
subject (Intercept) 2.0497e-02 0.1431662
Order1 4.1101e-06 0.0020273 -1.000
Voice1 1.3959e-02 0.1181500 0.808 -0.808
Order1:Voice1 2.0579e-02 0.1434552 0.290 -0.290 0.540
sentence (Intercept) 2.6765e-03 0.0517352
Order1 1.1696e-02 0.1081500 0.320
Voice1 5.0552e-03 0.0710997 0.275 -0.531
Order1:Voice1 1.1587e-02 0.1076410 -0.805 0.200 -0.293
Residual 4.6331e-01 0.6806653
Number of obs: 720, groups: subject, 36; sentence, 20
Fixed effects:
Estimate Std. Error t value
(Intercept) 0.26626 0.03667 7.261
Order1 -0.13508 0.03495 -3.865
Voice1 -0.11835 0.03568 -3.317
Correlation of Fixed Effects:
(Intr) Order1
Order1 0.066
Voice1 0.332 -0.175
anova(m1,mi)
Data: rawdata Models: mi: score ~ Order + Voice + (Order * Voice | subject) + (Order * mi: Voice | sentence) m1: score ~ Order * Voice + (Order * Voice | subject) + (Order * m1: Voice | sentence) Df AIC BIC logLik Chisq Chi Df Pr(>Chisq) mi 24 1622.6 1732.5 -787.29 m1 25 1611.8 1726.3 -780.90 12.78 1 0.0003504 *** --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 2014-03-03 18:22 GMT+01:00 Levy, Roger <rlevy at ucsd.edu>:
(cc-ing the list, which I forgot to do on the last response...) On Mar 3, 2014, at 8:41 AM, Emilia Ellsiepen <emilia.ellsiepen at gmail.com> wrote:
2014-03-03 16:23 GMT+01:00 Levy, Roger <rlevy at ucsd.edu>:
On Mar 3, 2014, at 3:09, "Emilia Ellsiepen" <emilia.ellsiepen at gmail.com> wrote: Thank you for your feedback and this interesting discussion.
First off, it is not clear that Emilia's specific problem is being caused by over-parameterization. Emilia, could you perhaps give more information about the nature of the dataset that you're analyzing? Is it a 2x2 within-subjects, within-sentence balanced design without a great deal of missing data? In my experience with the last few pre-1.0 versions, lme4 is generally very good at converging to an optimum for these kinds of datasets with the number of observations and groups your fitted model reports. Have you tried fitting the model with the nlminb optimizer, either by including optimizer="optimx",optCtrl=list(method="nlminb") in the list of arguments to lmerControl, or by using the last pre-1.0 version of lme4 (available as lme4.0 on R-Forge)? Do you still get similar problems with the nlminb optimizer? (You should definitely not get the result that the simpler model has a higher log-likelihood.)
The design was a balanced 2x2 with-in subjects and with-in sentences design without any missing data from a magnitude estimation experiment. When I use the nlminb optimizer (by installing the lme4.0 version), I do get the interaction using the likelihood-ratio test, but I also get the following warning message: Warning message: In mer_finalize(ans) : singular convergence (7)
Thanks for this follow-up, Emilia! Question: do you get this warning when you fit the more complex model (with the fixed-effects interaction), or the null-hypothesis model (without the fixed-effects interaction)?
That was for the more complex model with interaction. The one without interaction converged without problems.
Thanks Emilia. Could you please give us a bit more information -- show us the fitted models (both the null and alternative-hypothesis models), and also the results of the call to anova()? Best Roger