Improving computation time for a binary outcome in lme4
On Mon, 24 May 2010, Douglas Bates wrote:
If you use the verbose option to lmer and to glmer on similar problems you will see that lmer if optimizing over fewer parameters than is glmer.
Just one potentially useful observation: Turning on "verbose" makes the waiting period much MUCH more tolerable. It's kinda like a progress bar--you know glmer is doing something and that makes it easier to wait. For some huge models with bigger-than-I-needed data sets (back in the netflix prize days), I just let R run overnight and got what I wanted--but I had never let it go more than an hour before I worried that it was looping.
Are there any suggestions on what I can do (other than simplify the model) to improve the computation time for a binary outcome?
There are the usual suspects of getting access to a fast computer with lots of memory and a 64-bit operating system. You could see whether an accelerated BLAS will help. For example, Revolution R has the MKL BLAS built-in. Regrettably, that isn't always a speed boost. We have seen situations where multi-threaded BLAS actually slow down sparse matrix operations because the communications overhead is greater than the time savings of being able to perform more flops per second.
My kingdom for multi-threaded nlm()... --Adam