Skip to content

Help understanding residual variance

10 messages · Greg Snow, Ista Zahn, Reinhold Kliegl +3 more

#
Hi all,

I'm trying to understand what the residual variance in this model:

tmp <- subset(sleepstudy, Days == 1 | Days == 9)
m1 <- lmer(Reaction ~ 1 + Days + (1 + Days | Subject), data = tmp)
tmp$fitted1 <- fitted(m1)

represents. The way I read this specification, an intercept and a
slope is estimated for each subject. Since each subject only has two
measurements, I would expect the Reaction scores to be completely
accounted for by the slopes and intercepts. Yet they are not: the
Residual variance estimate is 440.278.

This is probably a stupid question, but I hope you will be kind enough
to humor me.

Best,
Ista
#
Yes, each person has their own slope and intercept estimated, however
the slope and intercept are not determined solely by the 2 data points
for that person, but also are affected by the slope and intercept
estimates across all subjects (this is why lmer gives value beyond
lmList).

You can see this if you refit using the nlme package (only because it
has the augPred function which has not been implemented in lme4 yet):

library(nlme)
m2 <- lme( Reaction ~ Days, data=tmp, random=~Days|Subject)
plot(augPred(m2, ~Days, level=c(0,1)))

comparing the m2 model to your m1 gives the same fixed effects, but
slightly different random effects (I probably did not do something
that was needed to make the models exactly the same) but is probably
close enough.

Look at the plot and you will see the fixed effects line, the line for
each subject that includes the random effects, and the data.  The line
for the individual subjects are pulled slightly towards the fixed
effects line and so does not hit the 2 points exactly.  This shows how
the estimate of each individuals values are influenced by the overall
fit.
On Mon, Mar 26, 2012 at 8:18 PM, Ista Zahn <istazahn at gmail.com> wrote:

  
    
#
Thank you Greg, that helps.

-Ista
On Tue, Mar 27, 2012 at 11:32 AM, Greg Snow <538280 at gmail.com> wrote:
1 day later
#
Hello again,

Sorry for bringing this up again. The thing is that a statistician
consulting with my research group insists that you cannot have both
random intercepts and random slopes when there are only two
observations per group. Clearly I can fit such a model using lmer(),
but this only serves to convince my local statistician that "R is
doing something strange". I suspect that this is a hopelessly vague
question, but is R doing something strange? Or is my statistician
incorrect in claiming that you can't fit both random intercepts and
random slopes with only two observations per group?

Again, I realize this is not a great question, but I would really
appreciate any thoughts on the matter.

Best,
Ista
On Tue, Mar 27, 2012 at 2:55 PM, Ista Zahn <istazahn at gmail.com> wrote:
#
But why is Greg Snow's response inadequate?

Restating his argument:  In an LMM we are not estimating individual
random effects (means, slopes) and individual residuals, but variance
of random effects and variance of residuals. So there can be
differences between a subject's observed random effect and random
slope  and conditional modes of the distribution of the random effects
(i.e., the point of maximum density), given the observed data and
evaluated at the parameter estimates.

I think your statistician's answer is a good argument that you must
not treat conditional modes as independent observations in a
subsequent analyses. For example, we showed with simulations that
correlations between conditional modes of slopes and intercepts are
larger than the correlation parameter estimated in the LMM (Kliegl,
Masson, & Richer, Visual Cognition, 2010).

Reinhold Kliegl

--
Reinhold Kliegl
http://read.psych.uni-potsdam.de/pmr2
On Tue, Mar 27, 2012 at 4:18 AM, Ista Zahn <istazahn at gmail.com> wrote:
#
Hi Reinhold,

Good question. My consultant didn't seem impressed when I tried to
articulate that explanation, but perhaps I wasn't clear.

Thanks,
Ista
On Thu, Mar 29, 2012 at 1:45 AM, Reinhold Kliegl
<reinhold.kliegl at gmail.com> wrote:
#
Hi Ista,

To me the onis is on the statistician consultant to explain *why* you
cannot have both random intercepts and slopes.  Does the consultant
have papers to reference or proofs?

In any case, this is hardly exclusive to 'R doing something strange'.
SAS and Stata happily join the gang.  See the attached file for code
and output from all three using a minidataset simulated in R.

I suppose one could bicker over whether a random intercept and slope
is a good idea, but possible it certainly is.  You might suggest that
it is poor fare to voice strong opinions about matters which one does
not understand.

Cheers,

Josh
On Thu, Mar 29, 2012 at 3:29 AM, Ista Zahn <istazahn at gmail.com> wrote:

  
    
#
Thanks Josh, the comparison with SAS and Stata is very useful. I'll
see what my consultant has to say about this example.

Best,
Ista
On Thu, Mar 29, 2012 at 11:29 AM, Joshua Wiley <jwiley.psych at gmail.com> wrote:
#
Joshua Wiley <jwiley.psych at ...> writes:
Very nice example.  Do note that while R::lme and Stata give
the same point estimates, Stata also provides estimated confidence
intervals, which are enormous -- suggesting that, while one can
do this, the resulting model might be unidentifiable.

  Doug Bates commented off-list that:
cheers
    Ben
#
On Mar 29, 2012, at 11:25 AM PDT, Ben Bolker wrote:

            
I concur with this -- I believe that the statistician consultant is right in this case, and that with only two observations per subject (plus crucially, no pair of observations for a given subject having the same value of Days), the model is actually unidentifiable given the dataset.  Here's one way of thinking about it: imagine that we represent the Days==1 and Days==9 observations for each subject as a 2-vector, (y_1 y_9).  If the residual variance is s^2 and the covariance matrix for the subject-level means for Days==1 and Days==9 as

 | t1^2             rho*t1t9     |
 | rho*t1t9             t9^2     |

then the marginal distribution for the total contribution of subject-level and residual error to the two observations for a given subject is bivariate normal with covariance matrix

 | t1^2 + s^2       rho*t1t9 |
 | rho*t1t9         t9^2+s^2 |

But as this illustrates, there's no way of distinguishing the residual error term s^2 from the subject random effects t1^2 and t9^2.  (This problem would not arise if there were at least two observations for one value of Days in at least one subject, because for that subject one would not be able to represent the data such that there is effectively only one multivariate observation per subject.)

I'll be interested to know whether this explanation helps shed light on the matter!

Best

Roger

--

Roger Levy                      Email: rlevy at ucsd.edu
Assistant Professor             Phone: 858-534-7219
Department of Linguistics       Fax:   858-534-4789
UC San Diego                    Web:   http://idiom.ucsd.edu/~rlevy