Skip to content

Summarizing the fitted model takes more RAM than

6 messages · Gorjanc Gregor, Douglas Bates, Gabor Grothendieck +1 more

#
I simply have three random effects. The model specification (without indexes) would be

y ~ N(mu, sigma^2_e)

mu = alpha + ... + rejec + hy + hys

rejec ~ N(0 sigma^2_r)

hy ~ N(0 sigma^2_hy)

hys ~ N(0 sigma^2_hys)

gg
#
On Mon, Dec 15, 2008 at 3:36 AM, Gorjanc Gregor
<Gregor.Gorjanc at bfro.uni-lj.si> wrote:
I think you mean "second level factor" not "two-level factor", Zhijie.
 I know it is confusing but "two-level factor" means a factor which
has only two possible values, such as sex which has only two levels -
male and female.  Two-level factorial designs, for example, are
specialized experimental designs where all the experimental factors
are restricted to only two levels.

I believe you are using the terminology of multilevel modeling where
one characterizes factors as being at the first level, the second
level, etc.  One can fit multilevel models using lmer but one can also
fit models that do not fit into the multilevel framework - models with
crossed or partially crossed random effects.  Unfortunately, because
the terminology of multilevel models (also called hierarchical linear
models or HLMs) is so pervasive, many people believe that any model
with random effects must be hierarchical.  That is not true and it
leads to many misconceptions.

Even worse, the emphasis on levels in model specification for software
like MLWin and HLM causes confusion about fixed effects and random
effects.  These is a widespread belief that the level of a variable
affects its use in the fixed-effects specification, which is not true.

Gorjanc is correct in what he writes below.  He has three random
effects terms.  They are what I would call simple, scalar random
effects ("scalar" because there is one random effect associated with
each level of the factors rejec, hy and hys and "simple" because the
term on the left side of the "|" is "1", which means that each term
contributes a set of indicator columns to the model matrix for the
random effects). Whether the factors rejec, hy and hys happen to form
a nested sequence is irrelevant to the model specification and fitting
in lmer.  In the multilevel world it is mandatory.
#
Note that the fitted method in lme had a level= argument that is no
longer available in lmer presumably because lmer does not assume
a hierarchy -- but do we have or will we have an easy way to get the
same effect as fitted(..., level=) in lmer?

library(nlme)
# example from plot.lme
fm1 <- lme(distance ~ age, Orthodont, random = ~ age | Subject)

fit0 <- fitted(fm1, level = 0)
fit1 <- fitted(fm1, level = 1)

(Maybe this is a bad example since its actually not so hard:
   fitted(lmer(distance ~ age + (age|Subject), Orthodont))
gives level 1 and
  fitted(lm(distance ~ age, Orthodont))
gives level 0 but even here it involved the complexity of using different
approaches to get them whereas with lme one could do it in a
unified manner.)
On Mon, Dec 15, 2008 at 10:19 AM, Douglas Bates <bates at stat.wisc.edu> wrote:
#
On Mon, Dec 15, 2008 at 10:16 AM, Gabor Grothendieck
<ggrothendieck at gmail.com> wrote:
One would need to define such a method carefully.  If the factors
defining random effects form a strictly nested sequence then there is
an interpretation of level.  If you do not have a strictly nested
sequence then I can only make sense of having all the random effects
defining fitted values (which is what the method for fitted returns
now) or having none of them.  The second is using the fixed-effects
coefficients only.
I don't know if fitted(lm(distance ~ age, Orthodont)) produces the
desired result. Removing the random effects from the prediction is not
always the same as removing the random effects from the fit.  I would
get the fitted values for the fixed effects only using

as.vector(model.matrix(fm1) %*% fixef(fm1))
#
On Mon, Dec 15, 2008 at 11:35 AM, Douglas Bates <bates at stat.wisc.edu> wrote:
They do seem to be the same here:
[1] TRUE

but your idea is better since it seems more general allowing at least
fixed vs. all effects which in many cases many be sufficient.  Some
easy way of specifying just the fixed effects might be nice as arg to fitted.

By the way, one other annoyance is that one cannot write:

# fixed effects only
lmer(distance ~ age, Orthodont)

which would be nice when comparing models so that one stays in the same
function, lmer, rather than switching from lmer to lm when the model changes
slightly, and so one is sure that the various outputs are in consistent forms.
1 day later