How to check if %dopar% really run parallel?
There is a mistake. Rather than:
times(10000) %dopar% fun
you should write:
times(10000) %dopar% fun()
On my machine, "fun" executes in about 0.4 seconds, so executing
it 10,000 times should take over an hour to execute. Your error turned
a real program into a toy program. The error also resulted in more
communication, since now the function itself is being returned by the
workers.
When I ran your benchmark on my machine with 100, rather than 10,000
tasks, I got the following results:
user system elapsed
43.573 0.191 43.823
user system elapsed
0.093 0.007 24.890
That's not so bad.
- Steve
On Tue, May 4, 2010 at 12:22 AM, Mario Valle <mvalle at cscs.ch> wrote:
Is there any way to check that %dopar% really runs parallel? The following code (on a dual core laptop running windows+R 2.11.0pat and on Linux+R2.11.0) runs %dopar% more slowly than the same %do% code. BTW, if you see any obvious mistake in the code... Thanks! ? ? ? ? ? ? ? ?mario library(doSNOW) library(foreach) fun <- function() for(q in 1:1000000) sqrt(3) system.time(times(10000) %do% fun, gcFirst = TRUE) # ? user ?system elapsed # ? 5.74 ? ?0.01 ? ?6.24 cl <- makeCluster(2, type = "SOCK") registerDoSNOW(cl) system.time(times(10000) %dopar% fun, gcFirst = TRUE) # ? user ?system elapsed # ? 7.89 ? ?0.19 ? ?9.01 stopCluster(cl) -- Ing. Mario Valle Data Analysis and Visualization Group ? ? ? ? ? ?| http://www.cscs.ch/~mvalle Swiss National Supercomputing Centre (CSCS) ? ? ?| Tel: ?+41 (91) 610.82.60 v. Cantonale Galleria 2, 6928 Manno, Switzerland | Fax: ?+41 (91) 610.82.82
_______________________________________________ R-sig-hpc mailing list R-sig-hpc at r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-hpc