lme4 performance
Regarding the slowness issue of lme4 vs. lme4.0 I have had some good initial results getting lme4 to run as fast or faster than lme4.0 by replacing the optimizer (which can be done without touching the code of lme4 itself thanks to lme4's modular nature). See my comment in lme4 issue 150 on github https://github.com/lme4/lme4/issues/150 where I have posted the results and the code need to try this. If anyone has a mixed model dataset they would like to try out using this approach please let me know. On Sun, Jun 1, 2014 at 11:04 PM, Gabor Grothendieck
<ggrothendieck at gmail.com> wrote:
Ben Bolker and I have been running some lmer timings on the lme4 package and the older lme4.0 package and I am posting some timings here for comment. This model was used: lmer(y ~ service * dept + (1|s) + (1|d), InstEval) with lme4 from https://github.com/lme4/lme4/ and lme4.0 from http://download.r-forge.r-project.org/src/contrib/lme4.0_0.999999-4.tar.gz The newer lme4 is slower than the older lme4.0: - lme4 takes 51 sec for the above command vs. 33 sec for lme4.0 - lme4 spends 37 sec of that in C/C++ optimizer whereas lme4.0 spends 26 sec - lme4 spends 33 sec of that performing cholesky factorization whereas lme4.0 spends 14 sec - lme4 involves 57 iterations of bobyqa wheresd lme4.0 involves 14 iterations of nlminb. The iterations are the counts displayed if the actual argument verbose = TRUE is added to the lmer call. We can reduce the 51 sec run time of lme4 to 45 sec (about 10% savings) if we add this actual argument to the lmer call above: control = lmerControl(calc.derivs = FALSE) The fixed and random coefficients from the two packages were very close. system.time(lmer(...)) was used to get the lmer run time. UNIX time was used to get the time used when launched from littler and the percentages given by the google perftools profiler, also launched from littler, were applied to the UNIX time to get the other timings. R 3.0.3 running under Linux was used. -- Statistics & Software Consulting GKX Group, GKX Associates Inc. tel: 1-877-GKX-GROUP email: ggrothendieck at gmail.com
Statistics & Software Consulting GKX Group, GKX Associates Inc. tel: 1-877-GKX-GROUP email: ggrothendieck at gmail.com