More naive questions: Speed comparisons? what is a "stack imbalance" in lmer? does lmer center variables?

  Source *is* available:

Peter Dalgaard wrote:

Ben Bolker wrote:

   As of sometime last year (? I think ?), ADMB is
free/gratis/libre/open source (BSD licensed). 

As of _next_ year is more like it. Their downloads are still 
binary-only. BSD does not imply Open Source when you don't have the 
sources, and "open source project" in this case means a project to make 
the software open source. The intention appears to be honourable, though.

  Even before that,

glmmADMB (which was an R package with a binary component, available for
download) was "free as in beer". In its current status, I think of ADMB
in the same category as WinBUGS -- a powerful, albeit sometimes
unwieldy, tool that can be used through an R interface to solve general
problems by writing model descriptions in a non-R language.
  I have to agree with Kevin that the diversity of mixed model software
tools is a good thing.

  cheers
    Ben Bolker

Douglas Bates wrote:

Got to disagree with you, Kevin.  admb and asreml are not part of R,
even in the general sense of R packages.  R is Open Source - they are
not. Tacking on an R interface to proprietary software and saying it
is available in R is misleading and dishonest.

On Wed, Sep 23, 2009 at 8:54 AM, Kevin Wright <kw.stat at gmail.com> wrote:

Paul,

It appears to me that the published timings you reference are
comparing the __nlme__ package with other software.  So the answer is
yes, nlme really is that slow for some models.  You are probably aware
that the __lme4__ package has faster algorithms.

There are many ways to fit mixed models in R including nlme, lme4,
MCMCglmm, admb asreml, BUGS, etc.  If I was teaching a course, I would
try to expose students to at least two of those in some detail and
touch briefly on the others: nlme can fit a variety of complex
varaiance structures, lme4 has faster algorithms, asreml is the only
choice of animal/plant breeders and has commercial support, MCMCglmm
has some Bayesian aspects and can fit some heteroskedastic variance
structures, admb is used in Fish & Wildlife, etc.

Mixed model fitting in R is definitely not a case of "one size fits all".

Kevin Wright


On Wed, Sep 23, 2009 at 1:36 AM, Paul Johnson <pauljohn32 at gmail.com> wrote:

Sent this to r-sig-debian by mistake the first time.  Depressing.

1.  One general question for general discussion:

Is HLM6 faster than lmer? If so, why?

I'm always advocating R to students, but some faculty members are
skeptical.  A colleague compared the commercial HLM6 software to lmer.
 HLM6 seems to fit the model in 1 second, but lmer takes 60 seconds.

If you have HLM6 (I don't), can you tell me if you see similar differences?

My first thought was that LM6 uses PQL by default, and it would be
faster.  However, in the output, HLM6 says:

Method of estimation: restricted maximum likelihood

But that doesn't tell me what quadrature approach they use, does it?

Another explanation for the difference in time might be the way HLM6
saves the results of some matrix calculations and re-uses them behind
the scenes.  If every call to lmer is re-calculating some big matrix
results, I suppose that could explain it.

There are comparisons from 2006 here

http://www.cmm.bristol.ac.uk/learning-training/multilevel-m-software/tables.shtml

that indicate that lme was much slower than HLM, but that doesn't help
me understand *why* there is a difference.

2. What does "stack imbalance in .Call" mean in lmer?

Here's why I ask.  Searching for comparisons of lmer and HLM,  I went
to CRAN &  I checked this document:

http://cran.r-project.org/web/packages/mlmRev/vignettes/MlmSoftRev.pdf

I *think* these things are automatically generated.  The version
that's up there at this moment  (mlmRev edition 0.99875-1)  has pages
full of the error message:

stack imbalance in .Call,

Were those always there?  I don't think so.   What do they mean?

3. In the HLM6 output, there is a message at the end of the variable list:

'%' - This level-1 predictor has been centered around its grand mean.
'$' - This level-2 predictor has been centered around its grand mean.

What effect does that have on the estimates?  I believe it should have
no effect on the fixed effect slope estimates, but it seems to me the
estimates of the variances of random parameters would be
changed.  In order to make the estimates from lmer as directly
comparable as possible, should I manually center all of the variables
before fitting the model?   I'm a little stumped on how to center a
multi-category factor before feeding it to lmer.  Know what I mean?

pj

--
Paul E. Johnson
Professor, Political Science
1541 Lilac Lane, Room 504
University of Kansas