How can I make R using more than 1 core (8 available) on a Ubuntu Rstudio server ?

@DB, I thought you were retired :)  But, to the OP, lme4 functions already take advantage of many computational methods that make computing these models to large data sets faster than (virtually) all other packages for estimating mixed linear models.

The packages you might come across for parallel processing won't necessarily apply here. For example, the foreach package is fantastic, but could not be applied to a glmer model.

Although, Doug, I do recall coming across some work I think in the Microsoft R distribution that did some parallel computing for matrix problems by default. I'm saying this by memory and cannot recall specifics.

With that said, I'm not certain parallel processing is the right thing to do with problems of this sort. Iteration t+1 depends on iteration t and when solutions to the problem live on a different processor, the expense of combining those things back together is not always faster, but instead can actually be even more expensive and slower.



-----Original Message-----
From: R-sig-mixed-models [mailto:r-sig-mixed-models-bounces at r-project.org] On Behalf Of Douglas Bates
Sent: Thursday, January 18, 2018 3:07 PM
To: Nicolas B?d?re <n.bedere at gmail.com>
Cc: R SIG Mixed Models <r-sig-mixed-models at r-project.org>
Subject: Re: [R-sig-ME] How can I make R using more than 1 core (8 available) on a Ubuntu Rstudio server ?

The procedure is fairly simple - just rewrite the lme4 package from scratch. :-)

I want to run the *glmer* procedure on a ?large? dataset (250,000 
observations). The model includes 5 fixed effects, 2 interactions 
terms and
3 random effects. It takes more than 15 min to run on my laptop 
(recent intel core i7, RAM = 4GO). Thus, the IT department of the 
University I am working at developed a Rstudio server based on the 
Ubuntu system. My problem is that 8 cores are available on this server 
but when I run the *glmer *procedure, only 1 of them is being used and 
it takes more than 1h to get the results... How can I solve that 
problem and improve time efficiency? I found on google I may have to 
use the parallel procedure but (i) I am not familiar at all with those 
informatics procedures and they look a bit complicated, (ii) the code 
I picked works with other functions in other packages such as 
*kmeans{stats}* (

https://stackoverflow.com/questions/29998718/how-can-i-make-r-use-more
-cpu-and-memory
)
but neither with *lmer *nor *glmer.*



Can you please help with a simple procedure to tackle the problem?


Many thanks !

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-mixed-models at r-project.org mailing list 
https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models

_______________________________________________
R-sig-mixed-models at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models