Hi, I just came across the following issue regarding mixed effects models: In a longitudinal study individuals (variable ind) are observed for some response variable. One explanatory variable, f, entering the model as fixed effect, is a (2-level) factor. The expression of that factor is constant for each individual across time (say, the sex of the individual). ind enters the model as grouping variable for random effects. So in a simple form, the formula could look like: y ~ f + ... + (1|ind) [and in the simplest model, the ellipsis is simply nothing] To me, this seems not to be an unusual design at all. However, the indicator matrix consisting of f and ind - say if ind had entered the model as fixed effect - shows a singularity. My question is now what will this 'singularity' cause in a mixed-effects model ? I admit, I have never fully understood how the fitting of mixed-effects models happen internally (whether REML or ML) [so I am not even sure if it can be called a 'singularity']. Specifically, does it make the fit numerically more unstable? Would the degree of this depend on other variables of the model? Is the issue of degrees of freedom - complicated enough anyway for mixed models - further inflated by that? Have statistical inferences regarding the fixed effect be treated more carefully? Is the general situation something that should be avoided ? many thanks in advance for any insights and cheers, Thomas
'singularity' between fixed effect and random factor in mixed model
3 messages · Thomas Mang, Daniel Malter, Douglas Bates
An Econometrician may help you with more theoretical insights, but you could do Monte-Carlo simulations of data and analyze the effects you are interested in. Daniel ------------------------- cuncta stricte discussurus ------------------------- -----Urspr?ngliche Nachricht----- Von: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] Im Auftrag von Thomas Mang Gesendet: Wednesday, July 01, 2009 7:53 PM An: r-help at stat.math.ethz.ch Betreff: [R] 'singularity' between fixed effect and random factor in mixedmodel Hi, I just came across the following issue regarding mixed effects models: In a longitudinal study individuals (variable ind) are observed for some response variable. One explanatory variable, f, entering the model as fixed effect, is a (2-level) factor. The expression of that factor is constant for each individual across time (say, the sex of the individual). ind enters the model as grouping variable for random effects. So in a simple form, the formula could look like: y ~ f + ... + (1|ind) [and in the simplest model, the ellipsis is simply nothing] To me, this seems not to be an unusual design at all. However, the indicator matrix consisting of f and ind - say if ind had entered the model as fixed effect - shows a singularity. My question is now what will this 'singularity' cause in a mixed-effects model ? I admit, I have never fully understood how the fitting of mixed-effects models happen internally (whether REML or ML) [so I am not even sure if it can be called a 'singularity']. Specifically, does it make the fit numerically more unstable? Would the degree of this depend on other variables of the model? Is the issue of degrees of freedom - complicated enough anyway for mixed models - further inflated by that? Have statistical inferences regarding the fixed effect be treated more carefully? Is the general situation something that should be avoided ? many thanks in advance for any insights and cheers, Thomas ______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
2 days later
On Thu, Jul 2, 2009 at 1:52 AM, Thomas Mang<thomas.mang at fiwi.at> wrote:
Hi,
I just came across the following issue regarding mixed effects models: In a longitudinal study individuals (variable ind) are observed for some response variable. One explanatory variable, f, entering the model as fixed effect, is a (2-level) factor. The expression of that factor is constant for each individual across time (say, the sex of the individual). ind enters the model as grouping variable for random effects. So in a simple form, the formula could look like: y ~ f + ... + (1|ind) [and in the simplest model, the ellipsis is simply nothing]
To me, this seems not to be an unusual design at all.
However, the indicator matrix consisting of f and ind - say if ind had entered the model as fixed effect - shows a singularity.
Yes.
My question is now what will this 'singularity' cause in a mixed-effects model ? I admit, I have never fully understood how the fitting of mixed-effects models happen internally (whether REML or ML) [so I am not even sure if it can be called a 'singularity'].
You do not encounter a singularity in solving for the conditional means of the random effects and the conditional estimates of the fixed effects because there is a penalty assigned to the size of the random effects vector. This removes the ill-conditioning of the least squares problem. It is sometimes called "regularization" of the estimation. Should you wish to find out what does go on inside the lmer function for REML or ML estimation of the parameters in a linear mixed model, you can check out the slides from a short course that I just finished at the University of Lausanne. Go to http://lme4.R-forge.R-project.org/slides and click on the link "2009-07-01-Lausanne". The display version of the slides for the theory section, 6TheoryD.pdf, is the best explanation I have yet been able to formulate for the theory. The important thing to note is that in the penalized linear least squares problem the predictions for the "pseudo-observations" are affected by the random effects but not by the fixed-effects.
Specifically, does it make the fit numerically more unstable? Would the degree of this depend on other variables of the model? Is the issue of degrees of freedom - complicated enough anyway for mixed models - further inflated by that? Have statistical inferences regarding the fixed effect be treated more carefully? Is the general situation something that should be avoided ? many thanks in advance for any insights and cheers, Thomas
______________________________________________ R-help at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.