How can I optimize the performance of mixed models?

Packages like parallel allow you to create temporary copies the current
R process such that the various copies can work on different problems
(using differnt cores).  Technically, that's similar to running several
instances of R in parallel, just more convenient because R automatically
collects the results of the parallel computations in the parent process. 

As you suspected, the execution of a single function like lmer can't be
split up with these packages.  Automatic parallelization of code written
for a single execution thread is a very hard problem.  In many cases
it's even impossible.  Therefore, in order to make use of several cores,
lmer would have to be rewritten in some way, which may also not be
trivial.  So, I'm afraid it's currently not possible to use several
cores and it's very likely going to stay that way for at least some
time.

For now, I can offer only two ideas: 1.) If you have to run several
models, e.g. for different dependent variables, you can use parallel to
fit these models concurrently, each on one core.  2.) If your models are
fit on very large data sets, it may happen that your computer is running
out of RAM.  In this case, the operating system will extend the RAM
using disk space.  The problem with that is that hard disks are several
orders of magnitude slower than RAM and therefore everything will slow
down to a crawl.  The solution is then to extend the RAM of your
computer.  RAM is cheap but you have to make sure that you have a 64 bit
operating system.  32 bit operating systems can't make use of RAM
capacities larger than 4 GB.  There is probably an upper limit to how
much RAM R can use but I don't know that from the top of my head.  If
your problem really is RAM, then extending RAM will give you a
tremendous speed-up.

Good luck!

  Titus

Felipe Vargas Reeve writes:

How can I optimize the performance of mixed models?

Thread (4 messages)