Why pure computation time in parallel is longer than the serial version?

________________________________________
Von: r-sig-hpc-bounces at r-project.org [r-sig-hpc-bounces at r-project.org]" im Auftrag von "Xuening Zhu [puddingnnn529 at gmail.com]
Gesendet: Samstag, 22. Februar 2014 11:30
An: Roger Bivand
Cc: r-sig-hpc at r-project.org
Betreff: Re: [R-sig-hpc] Why pure computation time in parallel is longer than the serial version?

Roger,
Much thanks to you~ I've done some further experiments to exploit something
like openblas for parallel.
My cpu is *Intel(R) Core(TM) i5-3210M CPU @ 2.50GHz.* There are 2 physical
cores and additional 2 logical cores. The memory size is 8G. And my
operating system is unbuntu 12.04.

I choose a 10^3 * 10^4 matrix and wants to evaluate its
multiplication(t(m)%*%m) time. I don't consider tcrossprod() because I just
want to make the computation longer. Maybe more cases can be compared later.

The R (3.0.2) is re-compiled with open-blas. Inline function is employed to
change the number of threads are defined as below:
require(inline)
openblas.set.num.threads <- cfunction( signature(ipt="integer"),
                                      body =
'openblas_set_num_threads(*ipt);',
                                      otherdefs = c ('extern void
openblas_set_num_threads(int);'),
                                      libargs = c
('-L/home/pudding/OpenBLAS/ -lopenblas'),
                                      language = "C",
                                      convention = ".C")
##################################################################################

First I only compare open-blas with default R BLAS in the experiment:
(1) multiplication with the default R BLAS:

mat = matrix(1:1e7,ncol=1e4)> system.time(t(mat)%*%mat)  user  system elapsed
84.517  0.320  85.090


(2) open-blas with 2 threads specified:

openblas.set.num.threads(2)
$ipt
[1] 2
system.time(t(mat)%*%mat)
  user  system elapsed
10.164   0.512   5.549

(3) open-blas with 4 threads specified:

openblas.set.num.threads(4)
$ipt
[1] 4

system.time(t(mat)%*%mat)
  user  system elapsed
26.954   1.556   8.147

Things is a little strange that 4 threads is even slower than 2 threads!

##################################################################
Then I want to mix multicore with open-blas. I try to change the implicit
parallel of matrix multiplication into explicit version. So I just split
the data into several partitions and  Things become very wired here.

(1) First I specify the number of threads to be 1 in open-blas, and 2 cores
are used in mclapply:
openblas.set.num.threads(1)
$ipt
[1] 1

system.time({
+   group = sample(rep(1:8,length.out=ncol(mat)))
+   mm = lapply(split(seq(ncol(mat)),group),function(i) mat[,i])
+   #mcaffinity(1:8)
+   #system(sprintf("taskset -p 0xffffffff %d", Sys.getpid()))
+   #cores = detectCores()
+   a = mclapply(mm,function(m){
+     cat('Running!!\n')
+     t(m)%*%m
+     #tcrossprod(m)
+   },mc.cores=2)
+   b = Reduce("+",a)
+ })
Running!!
Running!!
Running!!
Running!!
Running!!
Running!!
Running!!
Running!!
  user  system elapsed
 0.352   0.168   1.363

(2) Then I change cores in mclapply (in parallel package) from 2 to 4. The
time is even longer as below. (Sometimes it even gives me a segfault error
but unfortunately I have no means to reproduce it ><.)

system.time({
+   group = sample(rep(1:8,length.out=ncol(mat)))
+   mm = lapply(split(seq(ncol(mat)),group),function(i) mat[,i])
+   #mcaffinity(1:8)
+   #system(sprintf("taskset -p 0xffffffff %d", Sys.getpid()))
+   #cores = detectCores()
+   a = mclapply(mm,function(m){
+     cat('Running!!\n')
+     t(m)%*%m
+     #tcrossprod(m)
+   },mc.cores=4)
+   b = Reduce("+",a)
+ })
Running!!
Running!!
Running!!
Running!!
Running!!
Running!!
Running!!
Running!!
  user  system elapsed
 0.400   0.148   1.597

(3)  When I change the number of threads from 1 to some number>1, I can't
have results being returned from mclapply any more. There are some
conflicts between mclapply and open-blas multi-threads speeding algorithm,
I guess.


2014-02-18 18:38 GMT+08:00 Roger Bivand <Roger.Bivand at nhh.no>:

wesley goi <wesley at ...> writes:


Hi xuening,

I use multicore's mclapply() function extensively and have recently
changed the BLAS lib to openblas to help with running a PCA on a big
matrix, everything ran fine. However, I was wondering if the openblas
lib will interfere with multicore.

So i guess so far there? EURO (tm)s no way to assigned the threads which
openblas uses hence it shdnt be used in a multicore script to be
submitted to a cluster else it? EURO'll consume all the cores?

Please do use the list archives; the thread:

https://stat.ethz.ch/pipermail/r-sig-hpc/2012-April/001339.html

provides much insight into the AFFINITY issue - see also mcaffinity()
in the parallel package. If your BLAS is trying to use all available
cores anyway, and you then try to run in parallel on top of that, your
high-level processes will compete across the available cores for
resources with BLAS, as each BLAS call on each core will try to
spread work across the same set of cores. Please also see:

http://www.jstatsoft.org/v31/i01/

and perhaps also:

http://ideas.repec.org/p/hhs/nhheco/2010_025.html

Neither are new, but are based on trying things out rather than
speculating. As pointed out before, Brian's comment tells you what you
need to know:

https://stat.ethz.ch/pipermail/r-sig-hpc/attachments/20140213/662780c9/
attachment.pl

Hope this clarifies,

Roger


On 18 Feb, 2014, at 11:25 am, Xuening Zhu <puddingnnn529 at ...> wrote:

Hi Wesley:
I installed open-blas before. It went well when I run serial
operations. 2 threads can be seen in 'top'. But
I can't change thread number through the methods it provided.

_______________________________________________
R-sig-hpc mailing list
R-sig-hpc at r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-hpc




--
Xuening Zhu
--------------------------------------------------------
Master of Business Statistics
Guanghua School of Management, Peking University

       [[alternative HTML version deleted]]