Skip to content
Prev 703 / 20628 Next
Original Raw

autocorrelation

Douglas Bates
On Sat, Mar 29, 2008 at 2:52 PM, Alan Cobo-Lewis <alanc at umit.maine.edu> wrote:
So if you want a model with correlated error terms (in addition to the
correlation induced by the random effects) then you should use the
nlme package.
I don't recall any statements to the effect that the lme4 capabilities
would be a superset of the nlme capabilities.  It seems that whoever
made that decision should have informed me of it.

The development of the lme4 package has been generously funded by
several grants, the most important of which was an STTR contract that
we had for 3 years.  The purpose of that contract was to develop a
package that could fit generalized linear mixed models using the
Laplace approximation and allowing for crossed or partially crossed
grouping factors for the random effects.  The development is currently
funded by another grant specifically to provide for fitting models
with crossed and partially crossed random effects and with carryover
of random effects from one time period to another.

Neither generalized linear mixed models nor models with crossed or
partially crossed random effects can be fit (well without resorting to
egregious kludges) with the nlme package.  Even nonlinear mixed models
as fit by nlme are sub-optimal compared to the methods in lme4.  (lme4
uses direct optimization of the Laplace approximation to the
log-likelihood whereas nlme uses an alternating algorithm that Mary
Lindstrom and I proposed.)

My priorities are to fulfill the tasks that I proposed for these
grants and to build the best software that I can.  The beauty of open
source software is that if your priorities are different, you have
full access to the sources and you can modify them to fulfill your
objectives.  So I suggest that you
 - Continue to use the nlme package if you wish to incorporate
(additional) correlation structures in models
 - Design, code, test and document extensions to the lme4 package to
do so and then submit these changes as patches
 - Develop your own package so you can have things done the way you
want them to be done.  You already have access to the lme4 sources so
a lot of the heavy lifting has been done for you.

This developing the software is not as easy as it may seem.  There are
many trade-offs and, at least for me, it takes a lot of effort to
determine even if it is possible to incorporate various extensions
harmoniously.  It is possible to model the mean and variance of the
conditional distribution of the response separately when that
distribution is multivariate normal.  It is not as easy to do so when
that distribution is binomial or Poisson or some other distribution
for a generalized linear mixed model.  Because the first purpose of
the lme4 package was to allow for generalized linear mixed models I
did not incorporate (additional) correlation structures and variance
functions.  I'm not even sure it could be done consistently for GLMMs
but you are welcome to show us how.

Thread (3 messages)

Alan Cobo-Lewis autocorrelation Mar 29 Douglas Bates autocorrelation Mar 30 Alan Cobo-Lewis autocorrelation Mar 30