Hi all I'm looking for a way to specify a more complex variance-covariance matrix for the random effects in lmer(). For example, in lme() (nlme package), there are the pdMat classes to specify e.g. a compound symmetry (pdCompSym) or a multiple of an identity (pdIdent) structure for the variance-covariance matrix of random effects. Is there a possibility to code an equivalent for lmer()? I'm specifically interested in "compound symmetry" and "multiple of an identity". I assume that Douglas Bates or Ben Bolker have a distinct answer on that. Thanks in advance, Matthias --------------------------------------------------------- Matthias Suter, Dr.sc.nat. Forage Production and Grassland Systems Agroscope Reckenholzstrasse 191, CH-8046 Z?rich Phone +41 58 468 75 90 Fax +41 58 468 72 01 matthias.suter at agroscope.admin.ch www.agroscope.ch ---------------------------------------------------------
Random-effects variance-covariance matrix in lmer?
6 messages · Mollie Brooks, m@tthi@s@suter m@iii@g oii @groscope@@dmi@@ch, Phillip Alday +1 more
This is a frequent question for lme4 and MixedModels.jl and the answer
is....
it's not going to be supported in any mainstream release of either
package in the foreseeable future.
You can however still get some of these structures by clever
specification of the random effects.
For example:
data(Machines,package="nlme")
library("lme4")
m <- lmer(score ~ 1 + Machine + (0+Machine|Worker), Machines)
# with compound symmetry
m_cs <- lmer(score ~ 1 + Machine + (1|Worker) + (1|Worker:Machine),
Machines)
# pdDiag -- for continuous variables, you could just use the || syntax
m_diag <- lmer(score ~ 1 + Machine + (0+dummy(Machine,"A")|Worker) +
(0+dummy(Machine,"B")|Worker) + (0+dummy(Machine,"C")|Worker), Machines)
I'll leave pdIdent as an exercise for the OP. ;)
Best,
Phillip
On 02/10/2019 15:50, matthias.suter at agroscope.admin.ch wrote:
Hi all I'm looking for a way to specify a more complex variance-covariance matrix for the random effects in lmer(). For example, in lme() (nlme package), there are the pdMat classes to specify e.g. a compound symmetry (pdCompSym) or a multiple of an identity (pdIdent) structure for the variance-covariance matrix of random effects. Is there a possibility to code an equivalent for lmer()? I'm specifically interested in "compound symmetry" and "multiple of an identity". I assume that Douglas Bates or Ben Bolker have a distinct answer on that. Thanks in advance, Matthias --------------------------------------------------------- Matthias Suter, Dr.sc.nat. Forage Production and Grassland Systems Agroscope Reckenholzstrasse 191, CH-8046 Z?rich Phone +41 58 468 75 90 Fax +41 58 468 72 01 matthias.suter at agroscope.admin.ch www.agroscope.ch --------------------------------------------------------- [[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
There are additional random effect variance-covariance structures available in glmmTMB, including compound symmetry. They?re documented here https://cran.r-project.org/web/packages/glmmTMB/vignettes/covstruct.html <https://cran.r-project.org/web/packages/glmmTMB/vignettes/covstruct.html> cheers, Mollie
On 2Oct 2019, at 16:29, Phillip Alday <phillip.alday at mpi.nl> wrote:
This is a frequent question for lme4 and MixedModels.jl and the answer
is....
it's not going to be supported in any mainstream release of either
package in the foreseeable future.
You can however still get some of these structures by clever
specification of the random effects.
For example:
data(Machines,package="nlme")
library("lme4")
m <- lmer(score ~ 1 + Machine + (0+Machine|Worker), Machines)
# with compound symmetry
m_cs <- lmer(score ~ 1 + Machine + (1|Worker) + (1|Worker:Machine),
Machines)
# pdDiag -- for continuous variables, you could just use the || syntax
m_diag <- lmer(score ~ 1 + Machine + (0+dummy(Machine,"A")|Worker) +
(0+dummy(Machine,"B")|Worker) + (0+dummy(Machine,"C")|Worker), Machines)
I'll leave pdIdent as an exercise for the OP. ;)
Best,
Phillip
On 02/10/2019 15:50, matthias.suter at agroscope.admin.ch wrote:
Hi all I'm looking for a way to specify a more complex variance-covariance matrix for the random effects in lmer(). For example, in lme() (nlme package), there are the pdMat classes to specify e.g. a compound symmetry (pdCompSym) or a multiple of an identity (pdIdent) structure for the variance-covariance matrix of random effects. Is there a possibility to code an equivalent for lmer()? I'm specifically interested in "compound symmetry" and "multiple of an identity". I assume that Douglas Bates or Ben Bolker have a distinct answer on that. Thanks in advance, Matthias --------------------------------------------------------- Matthias Suter, Dr.sc.nat. Forage Production and Grassland Systems Agroscope Reckenholzstrasse 191, CH-8046 Z?rich Phone +41 58 468 75 90 Fax +41 58 468 72 01 matthias.suter at agroscope.admin.ch www.agroscope.ch --------------------------------------------------------- [[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
Thanks, Philipp. I was a aware of the pdDiag and the pdCorSym solution, and so I specifically asked for pdCompSym and pdIdent ... As for your compound symmetry suggestion, it is not quite clear to me. Up to now, I would have named "(1|Worker:Machine)" a random interaction term, specifying a distinct level for each worker*machine combination (and that's what is given by ranef(m_cs). How does this relate to the covariance in the compound symmetry, where also a multiple of identity is fit to diagonal? I have also tried to sort out the pdIdent equivalent, using similar coding as you suggest below, but was not successful so far. Do you know a similar solution? Thanks again, Matthias -----Urspr?ngliche Nachricht----- Von: Phillip Alday <phillip.alday at mpi.nl> Gesendet: Mittwoch, 2. Oktober 2019 16:30 An: Suter Matthias Agroscope <matthias.suter at agroscope.admin.ch>; r-sig-mixed-models at r-project.org Betreff: Re: [R-sig-ME] Random-effects variance-covariance matrix in lmer? This is a frequent question for lme4 and MixedModels.jl and the answer is.... it's not going to be supported in any mainstream release of either package in the foreseeable future. You can however still get some of these structures by clever specification of the random effects. For example: data(Machines,package="nlme") library(lme4) m <- lmer(score ~ 1 + Machine + (0+Machine|Worker), Machines) # with compound symmetry m_cs <- lmer(score ~ 1 + Machine + (1|Worker) + (1|Worker:Machine), Machines) # pdDiag -- for continuous variables, you could just use the || syntax m_diag <- lmer(score ~ 1 + Machine + (0+dummy(Machine,"A")|Worker) + (0+dummy(Machine,"B")|Worker) + (0+dummy(Machine,"C")|Worker), Machines) I'll leave pdIdent as an exercise for the OP. ;) Best, Phillip
On 02/10/2019 15:50, matthias.suter at agroscope.admin.ch wrote:
Hi all I'm looking for a way to specify a more complex variance-covariance matrix for the random effects in lmer(). For example, in lme() (nlme package), there are the pdMat classes to specify e.g. a compound symmetry (pdCompSym) or a multiple of an identity (pdIdent) structure for the variance-covariance matrix of random effects. Is there a possibility to code an equivalent for lmer()? I'm specifically interested in "compound symmetry" and "multiple of an identity". I assume that Douglas Bates or Ben Bolker have a distinct answer on that. Thanks in advance, Matthias --------------------------------------------------------- Matthias Suter, Dr.sc.nat. Forage Production and Grassland Systems Agroscope Reckenholzstrasse 191, CH-8046 Z rich Phone +41 58 468 75 90 Fax +41 58 468 72 01 matthias.suter at agroscope.admin.ch www.agroscope.ch --------------------------------------------------------- [[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
On 02/10/2019 16:52, matthias.suter at agroscope.admin.ch wrote:
Thanks, Philipp. I was a aware of the pdDiag and the pdCorSym solution, and so I specifically asked for pdCompSym and pdIdent ... As for your compound symmetry suggestion, it is not quite clear to me. Up to now, I would have named "(1|Worker:Machine)" a random interaction term, specifying a distinct level for each worker*machine combination (and that's what is given by ranef(m_cs). How does this relate to the covariance in the compound symmetry, where also a multiple of identity is fit to diagonal?
I stole this example from an old presentation of Doug Bates: http://pages.stat.wisc.edu/~bates/UseR2008/WorkshopD.pdf This is one of those things that my uni math instructors used to say "Convince yourself this is true". I hated it at the time, but I don't have the time to think about how to articulate this better than "thinking about what is being allowed in terms of geometry". Sorry. :(
I have also tried to sort out the pdIdent equivalent, using similar coding as you suggest below, but was not successful so far. Do you know a similar solution?
I don't off the top of my head. pdIdent really isn't relevant to my day job so I don't think about it much. I think it might be relatively straightforward to do in MixedModels.jl via something similar to the new zerocorr! function in the dev version, but it's not a priority for me. Pull-requests always welcome. (Slightly higher but still not terribly so on my priority list is documenting the internal fields of ReMat structure to make the logic of such modifications somewhat more obvious.) Phillip
Thanks again, Matthias -----Urspr?ngliche Nachricht----- Von: Phillip Alday <phillip.alday at mpi.nl> Gesendet: Mittwoch, 2. Oktober 2019 16:30 An: Suter Matthias Agroscope <matthias.suter at agroscope.admin.ch>; r-sig-mixed-models at r-project.org Betreff: Re: [R-sig-ME] Random-effects variance-covariance matrix in lmer? This is a frequent question for lme4 and MixedModels.jl and the answer is.... it's not going to be supported in any mainstream release of either package in the foreseeable future. You can however still get some of these structures by clever specification of the random effects. For example: data(Machines,package="nlme") library(lme4) m <- lmer(score ~ 1 + Machine + (0+Machine|Worker), Machines) # with compound symmetry m_cs <- lmer(score ~ 1 + Machine + (1|Worker) + (1|Worker:Machine), Machines) # pdDiag -- for continuous variables, you could just use the || syntax m_diag <- lmer(score ~ 1 + Machine + (0+dummy(Machine,"A")|Worker) + (0+dummy(Machine,"B")|Worker) + (0+dummy(Machine,"C")|Worker), Machines) I'll leave pdIdent as an exercise for the OP. ;) Best, Phillip On 02/10/2019 15:50, matthias.suter at agroscope.admin.ch wrote:
Hi all I'm looking for a way to specify a more complex variance-covariance matrix for the random effects in lmer(). For example, in lme() (nlme package), there are the pdMat classes to specify e.g. a compound symmetry (pdCompSym) or a multiple of an identity (pdIdent) structure for the variance-covariance matrix of random effects. Is there a possibility to code an equivalent for lmer()? I'm specifically interested in "compound symmetry" and "multiple of an identity". I assume that Douglas Bates or Ben Bolker have a distinct answer on that. Thanks in advance, Matthias --------------------------------------------------------- Matthias Suter, Dr.sc.nat. Forage Production and Grassland Systems Agroscope Reckenholzstrasse 191, CH-8046 Z rich Phone +41 58 468 75 90 Fax +41 58 468 72 01 matthias.suter at agroscope.admin.ch www.agroscope.ch --------------------------------------------------------- [[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
6 days later
It can be hacked with a little bit of effort: I just added a document describing this here: https://bbolker.github.io/mixedmodels-misc/notes/varmats.html (also see the .rmd version of the same thing).
On 2019-10-02 9:50 a.m., matthias.suter at agroscope.admin.ch wrote:
Hi all I'm looking for a way to specify a more complex variance-covariance matrix for the random effects in lmer(). For example, in lme() (nlme package), there are the pdMat classes to specify e.g. a compound symmetry (pdCompSym) or a multiple of an identity (pdIdent) structure for the variance-covariance matrix of random effects. Is there a possibility to code an equivalent for lmer()? I'm specifically interested in "compound symmetry" and "multiple of an identity". I assume that Douglas Bates or Ben Bolker have a distinct answer on that. Thanks in advance, Matthias --------------------------------------------------------- Matthias Suter, Dr.sc.nat. Forage Production and Grassland Systems Agroscope Reckenholzstrasse 191, CH-8046 Z?rich Phone +41 58 468 75 90 Fax +41 58 468 72 01 matthias.suter at agroscope.admin.ch www.agroscope.ch --------------------------------------------------------- [[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models