ML vs. REML to find a parsimonious mixed model
My take is that it usually doesn't matter and you can tell when it's likely to before you've started running models based on sample size. In practice, I almost always use ML unless I've got some specific reason to try REML (either I have sample size issues or it is for pedagogical purposes). REML will return less downwardly biased random effects so long as your sample size is small so you might as well do it if it matters. I've only seen them return materially different results in a couple of instances when it wasn't a toy data set designed to produce differences. If you are getting different p values for the deviance tests on random effects between REML and ML then you should probably take that as a sign to be extra paranoid. Most of my models are nonlinear so REML isn't strictly an option in a lot of software anyway (it is possible but often not on offer). I say do what you want as long as the deviance tests are still viable. On Mon, Apr 23, 2018 at 5:38 PM, Maarten Jung <
Maarten.Jung at mailbox.tu-dresden.de> wrote:
Hi Christoph, No, I didn't. And I'm still very interested in what other mixed model experts/experienced mixed model users think about it. At the moment I tend to use REML for this purpose. Best, Maarten On Mon, Apr 23, 2018 at 4:20 PM, Christoph Huber < christoph.huber-huber at univie.ac.at> wrote:
Hi Maarten, Did you get any responses yet? I was facing the same problem and went for REML eventually. But it still seems to me that this question does not
(yet)
have a definite answer. Best, Christoph Am 16.04.2018 um 12:00 schrieb r-sig-mixed-models-request at r-project.org: Send R-sig-mixed-models mailing list submissions to r-sig-mixed-models at r-project.org To subscribe or unsubscribe via the World Wide Web, visit
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models or, via email, send a message with subject or body 'help' to r-sig-mixed-models-request at r-project.org You can reach the person managing the list at r-sig-mixed-models-owner at r-project.org When replying, please edit your Subject line so it is more specific than "Re: Contents of R-sig-mixed-models digest..." Today's Topics: 1. ML vs. REML to find a parsimonious mixed model (Maarten Jung) ---------------------------------------------------------------------- Message: 1 Date: Sun, 15 Apr 2018 13:00:08 +0200 From: Maarten Jung <Maarten.Jung at mailbox.tu-dresden.de> To: Help Mixed Models <r-sig-mixed-models at r-project.org> Subject: [R-sig-ME] ML vs. REML to find a parsimonious mixed model Message-ID: <CAHr4Dycsa1wmOXKKmDuGzrQi8pxgXq55iQxjEoEzFvyYNmvUvA at mail.gmail.com> Content-Type: text/plain; charset="utf-8" I want to use LRTs via anova() on fitted linear mixed models (merMod objects) to find a parsimonious mixed model containing only variance components supported by the data (e.g. Matuschek et al. 2017 [1], Bates
et
al. 2015 [2]). In this situation my focus is *only on the reduction of the random
effects
part* of the models. The aforementioned papers use ML instead of REML estimation within this process. Douglas Bates seems to prefer ML model comparison due to the skewed nature of the distribution of variance estimators [3] and the user Wolfgang states that "the ML estimator usually has lower mean-squared
error
(MSE) than the REML estimator" [4]. However, literally every textbook I know suggests using REML estimation when comparing mixed models that
differ
only in their random effect parts. What would you suggest in this particular situation? ML or REML? Best regards, Maarten [1] https://arxiv.org/abs/1511.01864 [2] https://arxiv.org/abs/1506.04967 [3] https://stat.ethz.ch/pipermail/r-sig-mixed-models/2015q3/023750.html [4] https://stats.stackexchange.com/a/48770 [[alternative HTML version deleted]] ------------------------------ Subject: Digest Footer
_______________________________________________ R-sig-mixed-models mailing list R-sig-mixed-models at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models ------------------------------ End of R-sig-mixed-models Digest, Vol 136, Issue 26 *************************************************** ? Dr. Christoph Huber-Huber Center for Mind/Brain Sciences (CIMeC) University of Trento Corso Bettini 31 <https://maps.google.com/?q=Corso+Bettini+31+38068+
Rovereto&entry=gmail&source=g>
38068 Rovereto <https://maps.google.com/?q=Corso+Bettini+31+38068+
Rovereto&entry=gmail&source=g>
(TN), Italy e-mail: christoph.huberhuber at unitn.it
[[alternative HTML version deleted]]
_______________________________________________ R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
Thanks, John John Poe, Ph.D. Postdoctoral Scholar / Research Methodologist Center for Public Health Services & Systems Research University of Kentucky www.johndavidpoe.com [[alternative HTML version deleted]]