On 14-02-28 11:27 AM, Douglas Bates wrote:
On Fri, Feb 28, 2014 at 10:04 AM, Emilia Ellsiepen <
emilia.ellsiepen at gmail.com> wrote:
Dear list members,
in analyzing a data set using lmers with maximal random effect
structure and subsequent likelihood-ratio tests (LRTs) following Barr
et al. 2013, I ran into the following problem: In some of the LRTs, it
turned out that the simpler model (only main effects) has a higher
likelihood than the more complicated model (including interaction),
resulting in Chi=0. If I simplify the models by taking out the
interactions in the two random effect terms, the LRT for the
interaction has a highly significant result.>
It shows that the more complex model has not converged to the optimum
parameter values. This can be because the optimizer being used is a bad
choice (in recent versions of the lme4 package the default was a
Nelder-Mead optimizer that can declare convergence to values that are not
the optimal values) or it can be because the model is too complex. We say
that such models are over-parameterized.
This is why the Barr et al. advice is dangerous. In model selection there
are two basic strategies: forward and backward. Forward selection starts
with a simple model and adds terms until they are no longer justified.
Backward selection starts with the most complex model and drops terms. It
is well known that backward selection is problematic when you can't fit the
most complex model. Yet Barr et al. say unequivocally that you must use
backward selection. The result will be that many researchers, especially
in linguistics, will encounter these problems of complex models providing
worse fits than simpler models.
I wish that Barr et al. would have provided software that is guaranteed to
fit even the most complex model to a global optimum when they stated their
"one size fits all" strategy. they didn't and those with experience in
numerical optimization can tell you why. It is not possible to guarantee
convergence to an optimum in over-parameterized models.
[snip]
I would be very interested to know whether the more thorough
convergence tests that we have implemented in the development version of
lme4 would correctly report that there are convergence problems with the
model containing the maximal RE structure ...
Ben Bolker