How to check if %dopar% really run parallel?
*BIG RED FACE* I'm ashamed of myself, that's was the error! A small, stupid pair of parenthesis missing. Now the parallel version is faster than the serial one as it should. (serial: 57.41, parallel 2 cores: 39.31) Thanks to Stephen and all. mario
Stephen Weston wrote:
There is a mistake. Rather than:
times(10000) %dopar% fun
you should write:
times(10000) %dopar% fun()
On my machine, "fun" executes in about 0.4 seconds, so executing
it 10,000 times should take over an hour to execute. Your error turned
a real program into a toy program. The error also resulted in more
communication, since now the function itself is being returned by the
workers.
When I ran your benchmark on my machine with 100, rather than 10,000
tasks, I got the following results:
user system elapsed
43.573 0.191 43.823
user system elapsed
0.093 0.007 24.890
That's not so bad.
- Steve
On Tue, May 4, 2010 at 12:22 AM, Mario Valle <mvalle at cscs.ch> wrote:
Is there any way to check that %dopar% really runs parallel?
The following code (on a dual core laptop running windows+R 2.11.0pat and on
Linux+R2.11.0) runs %dopar% more slowly than the same %do% code.
BTW, if you see any obvious mistake in the code...
Thanks!
mario
library(doSNOW)
library(foreach)
fun <- function() for(q in 1:1000000) sqrt(3)
system.time(times(10000) %do% fun, gcFirst = TRUE)
# user system elapsed
# 5.74 0.01 6.24
cl <- makeCluster(2, type = "SOCK")
registerDoSNOW(cl)
system.time(times(10000) %dopar% fun, gcFirst = TRUE)
# user system elapsed
# 7.89 0.19 9.01
stopCluster(cl)
--
Ing. Mario Valle
Data Analysis and Visualization Group |
http://www.cscs.ch/~mvalle
Swiss National Supercomputing Centre (CSCS) | Tel: +41 (91) 610.82.60
v. Cantonale Galleria 2, 6928 Manno, Switzerland | Fax: +41 (91) 610.82.82
_______________________________________________ R-sig-hpc mailing list R-sig-hpc at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-hpc
Ing. Mario Valle Data Analysis and Visualization Group | http://www.cscs.ch/~mvalle Swiss National Supercomputing Centre (CSCS) | Tel: +41 (91) 610.82.60 v. Cantonale Galleria 2, 6928 Manno, Switzerland | Fax: +41 (91) 610.82.82