predict in lmer()?

On Sun, Jul 31, 2011 at 11:01 AM, David Winsemius
<dwinsemius at comcast.net> wrote:

On Jul 31, 2011, at 9:41 AM, Ewart Thomas wrote:

david, i'm now slogging thru the most informative thread!

as dennis advised, one has to do the 'predict' part 'by hand' when
using lmer.  this is done in the 2 lines:
mm = model.matrix(terms(fm1),newdat)
newdat$distance = mm %*% fixef(fm1)
good luck - this thing is worth sticking with!

The ongoing quest for prediction and confidence intervals is
chronicled in the mixed-effects Archive. The online discussion of
prediction intervals in lme4-derived models, i.e. objects of class
"mer" can be extracted with a Google search of the mixed-models
Archive at the "advanced search" page:

http://www.google.com/search?q=lme4+OR+mer+%22prediction+intervals%22+site%3Ahttps%3A%2F%2Fstat.ethz.ch%2Fpipermail%2Fr-sig-mixed-models%2F&hl=en&num=50

The other place to look is in Bates' drafts of the book he is writing
on lme4 methods. In chapter 1 of a 2010 draft he suggests plotting
prediction intervals (of the random effects)  thusly:

dotplot(ranef(fm1ML,postVar=TRUE))

And he continues using that method for the next few chapters (during
that year). I wonder if it makes sense to construct prediction
intervals without proper consideration of the fact that some of the
variability is assumed to arise from fixed effects. In his written
material it appears that Bates studiously avoids mixing random and
fixed effects in what he is calling "prediction intervals". There is
some discussion of this problem in his chapter 5 of the March 2011
material but the matrix math is too complex for me to follow and he
has no accompanying R code to go along with it.

You're mixing two concepts.

 The intervals to which you refer are not
prediction intervals on future responses. They are intervals that
contain the central 1-\alpha area under the (marginal) density curve
for each component in the conditional distribution of the
random-effects given the observed data and evaluated at the parameter
estimates.  So the random effects are an unobserved vector-valued
random vector.  The model is defined by the unconditional distribution
of the random effects and the conditional distribution of the
responses, given a value of the random effects.  From these two we can
derive the joint distribution of the random effects and the responses.
When we condition on the observed value of the responses we get a
conditional density of the random effects, given the observed data.
For a linear mixed model this is a multivariate Gaussian distribution.