Why pure computation time in parallel is longer than the serial version?

When doing calculations in parallel, there's also some overhead costs. If
computation time per core is short, the overhead costs may exceed the
computation time, time-wise raising the cost of parallel task.

Cheers,
Roman


On Thu, Feb 13, 2014 at 5:26 AM, Xuening Zhu <puddingnnn529 at gmail.com>wrote:

I am learning about parallel computing in R , and I found this happening in
my experiments.

Briefly, in the following example, why are most values of user time in
t smaller
than that in mc_t ? My machine has 32GB memory, 2 cpus with 4 cores and 8
hyper threads in total. Tools such as BLAS to enhance performance aren't
installed as well.

system.time({t = lapply(1:4,function(i) {
     m = matrix(1:10^6,ncol=100)
     t = system.time({
         m%*%t(m)
     })
     return(t)})})


library(multicore)
system.time({
     mc_t = mclapply(1:4,function(m){
         m = matrix(1:10^6,ncol=100)
         t = system.time({
             m%*%t(m)
         })
         return(t)
     },mc.cores=4)})

t[[1]]

user  system elapsed

11.136   0.548  11.703

[[2]]
user  system elapsed

11.533   0.548  12.098

[[3]]
user  system elapsed

11.665   0.432  12.115

[[4]]
user  system elapsed

11.580   0.512  12.115

mc_t[[1]]

user  system elapsed

16.677   0.496  17.199

[[2]]
user  system elapsed

16.741   0.428  17.198

[[3]]
user  system elapsed

16.653   0.520  17.198

[[4]]
user  system elapsed

11.056   0.444  11.520

mc_t and t measures pure computation time according to my comprehension.
  Such things happens to parLapply in parallel package as well. The memory
in my machine is enough for that computation. (It just takes a few percent
of that).

Also I try to run 4 similar Rscript as below by hand using command
'Rscript' at the same time on the same machine and save the results. The
elapsed time for each of them is about 12s as well. So I think it may not
be the contention of cores.

system.time({t = lapply(1,function(i) {
     m = matrix(1:10^6,ncol=100)
     t = system.time({
         m%*%t(m)
     })
     return(t)})})

So what happened during the parallel? Does mc_t really measure the pure
computation time? Can someone explain the whole process by step in detail?

Thanks.

--

Xuening Zhu

         [[alternative HTML version deleted]]