Why pure computation time in parallel is longer than the serial version?
wesley goi <wesley at ...> writes:
Hi xuening, I use multicore's mclapply() function extensively and have recently changed the BLAS lib to openblas to help with running a PCA on a big matrix, everything ran fine. However, I was wondering if the openblas lib will interfere with multicore. So i guess so far there???s no way to assigned the threads which openblas uses hence it shdnt be used in a multicore script to be submitted to a cluster else it??'ll consume all the cores?
Please do use the list archives; the thread: https://stat.ethz.ch/pipermail/r-sig-hpc/2012-April/001339.html provides much insight into the AFFINITY issue - see also mcaffinity() in the parallel package. If your BLAS is trying to use all available cores anyway, and you then try to run in parallel on top of that, your high-level processes will compete across the available cores for resources with BLAS, as each BLAS call on each core will try to spread work across the same set of cores. Please also see: http://www.jstatsoft.org/v31/i01/ and perhaps also: http://ideas.repec.org/p/hhs/nhheco/2010_025.html Neither are new, but are based on trying things out rather than speculating. As pointed out before, Brian's comment tells you what you need to know: https://stat.ethz.ch/pipermail/r-sig-hpc/attachments/20140213/662780c9/ attachment.pl Hope this clarifies, Roger
On 18 Feb, 2014, at 11:25 am, Xuening Zhu <puddingnnn529 at ...> wrote:
Hi Wesley: I installed open-blas before. It went well when I run serial
operations. 2 threads can be seen in 'top'. But I can't change thread number through the methods it provided.