Random or Fixed effects appropriate?
I think this is a reasonable summary.
You were not clear on how you plan to use the conditional modes (i.e.,
your point 1). Please keep in mind that conditional modes are not
independent "observations" like a group mean or within-group effect or
slope, simply because shrinkage correction uses all data. Also, for
example, their correlations (i.e., between intercept and x for units
of C) are typically not identical to the estimated model correlations
displayed in the random-effects part (see also the Bates quote in my
last comment).
In analyses of reaction times (using subjects and items as crossed
random factors; carried out with Mike Masson and Eike Richter, 2007),
model-based estimates of correlations among random effects revealed
"clearer" patterns than the correlations between means and effects
computed for each subject (as they should, given that they were
corrected for unreliability). Unlike for fixed-effects estimates,
however, estimates of correlations among random effects were quite
susceptible to violations of distributional assumptions for the
residuals--up to a change in the sign of the correlation! As far as
the use of conditional modes is concerned, the absolute values of
correlations between conditional modes were always larger than the
corresponding model estimates.
In simulations, the model estimates of correlations recovered the
"true" variances and correlations, even after random deletion of 50%
of the data, but the variance of the conditional modes always
underestimated the true variance and the difference between model
estimate and correlation based on conditional modes increased with the
absolute magnitude of the correlation. In other words, conditional
modes underestimated the variance and exaggerated covariances and
correlations of random effects in these simulations. The shrinkage in
variance reflects the contribution of the likelihood in the
computation of the conditional modes. In summary, according to these
simulations, the model estimates of correlations among random effects
are fine; the computed correlations based on conditional modes may
serve a useful heuristic function for further analyses but must be
handled with care.
Best
Reinhold
On Wed, Apr 9, 2008 at 11:21 AM, Nick Isaac <njbisaac at googlemail.com> wrote:
Dear all, Thanks for the comments and apologies for not providing more information. I (mis)judged it would be better to discuss the issue abstractly. There should be enough levels to estimate the variance of C and at least one other random effect: Number of obs: 1242, groups: D, 269; C, 64; B, 8; A, 3 My interpretation of comments by all three respondents is as follows: 1) extracting the random effects/BLUPs/conditional modes is reasonable in general 2) a taxonomy might be considered fixed or random, depending on the question and the number of units/levels 3) In my case, it would be better to use the conditional modes for x|C than to fit x*C as an interaction term. Best wishes, Nick On 08/04/2008, Andrew Robinson <A.Robinson at ms.unimelb.edu.au> wrote:
> On Tue, Apr 08, 2008 at 07:10:16PM +0200, Reinhold Kliegl wrote:
> > > My dataset has one continuous normally-distributed fixed effect and > > > four random effects that are nested (in fact, it is a taxonomy). For > > > simplicity, I've removed the variable names, so the dataset has the > > > following structure: > > > > > > y ~ x | A/B/C/D
> > It would be good to know how many units/levels you have for each of > > your four random effects. Those with fewer than, say, five, are good > > candidates for being specified as fixed effects. Think how many > > observations you need to get a stable estimate of a variance! > >
> > > lmer( y ~ x + (1|A) + (1|B) + (1|C) + (1|D) + C + x:C) #error: > > > Downdated X'X is not positive definite, 82
> > You cannot include C both as a random and a fixed effect
> > > > I do not believe that this is generally true. See, for example, >
> > require(lme4) > > (fm1 <- lmer(Reaction ~ Days + Subject + (Days|Subject), sleepstudy))
> > Therefore I am uncertain as to how you can draw this conclusion > without more information about the design (which the poster really > should have provided). > > >
> > > lmer( y ~ x + (1|A) + (1|B) + (1|C) + (1|D) + x:C) #gives sensible results
> > If this gives sensible results, I suspect you have very few levels of > > C, say, 2 or 3? > > In this case, definitely specify C and x and their interaction as > > fixed effects, e.g.: > > lmer( y ~ x*C + (1|A) + (1|B) + (1|D) > > > > The following may not apply to your case, but it might: Sometimes > > people think that a nested/taxonomic design implies a random effect > > structure (e.g., schools, classes, students). This is not true. If you > > have only a few units for each factor, you are better off to specify > > it as a fixed-effects rather than a random-effects taxonomy. (Of > > course, you lose generalizability, but if you want this you should > > make sure you have sample that provides a basis for it.)
> > > I can see the sense behind this position but sometimes a few units are > all that is available, and including them in a model as fixed effects > muddies the statistical waters, especially if they are the kinds of > effects that a model user will be unlikely to naturally condition upon. > > I do agree that if there are problems with model fitting and/or > interpretation when the design is rigorously followed, then a more > flexible approach can and should be adopted, and appropriate > allowances must be made. > >
> > The interpretation of conditional modes (formerly knowns as BLUPs, > > that is "predictions") is a tricky business, especially with few > > units per levels.
> > > Sorry, I think I've missed something. In what sense are the > conditional modes formerly known as BLUPs? > > Andrew > > > -- > Andrew Robinson > Department of Mathematics and Statistics Tel: +61-3-8344-6410 > University of Melbourne, VIC 3010 Australia Fax: +61-3-8344-4599 > http://www.ms.unimelb.edu.au/~andrewpr > http://blogs.mbs.edu/fishing-in-the-bay/ >