Skip to content
Prev 66176 / 398502 Next

slow computation of mixed ANOVA using aov

"Steven Lacey" <slacey at umich.edu> writes:
[You mean subject/(f1*f3), right? "f2" is a coarsening of "subject"]
Yes, this is basically correct. The main issue is the calculation of
the projection onto the terms in the Error model, which is done using
the generic lm algorithm even though the design is typicaly orthogonal
so that there are much faster ways to get to the result. 

To paraphrase: if you have double determinations and an error term at
the level of each pair, the algorithm fits a model with N/2 parameters
in order to calculate the mean and difference within each pair. For
large designs, this becomes an issue...

This is in some sense a tradeoff of generality for speed, but the
results for non-orthogonal designs are typically very hard to
interpret.

The topic does come up off and on, and we have been considering the
option of adding an algorithm where it is assumed that the Error model
consists of mutually orthogonal and balanced terms (in the sense that
all factor levels appear equally frequently). Just a "simple matter of
programming"... 

For the near term, there are a couple of things that you can do:

- avoid having an error term that is equivalent to the full data set.
  In your case, subject:f1:f3 is such a term, and subject/(f1+f3) is
  actually equivalent (the second order interaction term becomes the
  residual stratum). This at least saves you from inverting an NxN
  matrix. 

- use a version of R compiled with a fast BLAS, on a fast computer
  with a lot of RAM... (A ~2K square matrix inversion will take a
  while, but "hours" sounds a bit excessive). 

- (not too sure of this, but R 2.1.0 will have some new code for
  multivariate lm, with intra-individual designs, and
  tests under the sphericity assumptions; it is possible that
  your models can be reformulated in that framework. You'd have to
  set it up as a model with 160 responses on 56 subjects, though, and
  the code wasn't really designed with that sort of intrasubject
  dimensionality in mind.)