zero random effect sizes with binomial lmer
Daniel Ezra Johnson <johnson4 <at> babel.ling.upenn.edu> writes:
If one compares the random effect estimates, in fact, one sees that they are in the correct proportion, with the expected signs. They are just approximately eight orders of magnitude too small. Is this a bug?
BLUPs are essentially shrinkage estimates, where shrinkage is determined with magnitude of variance. Lower variance more shrinkage towards the mean - zero in this case. So this is not a bug. Gregor
I doubled each data set by duplicating each Subject. There are now 46 subjects in each group instead of 23. So now A and B differ in 2 observations out of 322. The lmer resuls are sensible.
Why would you want to do that? Doubling your data set is not what you want as you are hacking the analysis. You should definitely avoid this for real analysis!
I know BLUPs are not like regular parameters, but doubling (or cutting in half) the data should not, in my opinion, cause this behavior. There is a lot of room for the original A variance estimate to be lower than B, maybe it should be 0.05, 0.01, 0.05, or whatever, but not < .0000000005.
This is numerical procedure and if log-likelihood is flat then it might happen that algorithms give you such estimates. When you doubled the data log-likelihood gets more peaked shape and then it seems reasonable that estimates are more consistent.
I understand that the between-Item variance is low, and probably it is no greater than what you would expect to occur by chance, but isn't that what hypothesis testing is for (anova, etc.)?
Yes, but "anova etc." is not a super tool. If you get parameter estimates that are essentially 0, do you still need hypothesis testing?
Is my best way around the algorithm returning zero to do what I have done above, with my real data? That is, duplicate (or triplicate) Subjects to increase my data size, and thereby get a reasonably comparable (if wrong) estimate of the Item variance? Zero is not a reasonable estimate in any of these data sets.
No! Try with mcmcsamp and then you might get better view of posterior distribution of item variance. I think that MLE is near zero with some support for positive values. Try to play with lmer i.e. with starting values etc. Additionally, why do you want to estimate variances separately for dataset A and B. Can you really suggest that they should be different between datasets? Gregor