?I want to take advantage of my multicore CPU to speed up a loop in a simulation program.? I didn?t write the code, but the iterations appear independent to me, at least in the sense that the results of one loop do not depend on previous ones.? Right now I?m relegated to a Windows box that runs Windows 7.? These appear to be the options: ? Pnmath-appears to parallelize non-BLAS routine but requires a special build Fork-UNIX only Romp-looks like this hasn?t advanced past the developmental stage Multicore-use on Windows at your own risk R/parallel-seems like the best option if I don?t want to recompile. ? Has anyone ever used R/parallel?? What kind of results did you have?? One difficulty with my simulation is that the loop includes code to generate random numbers.? If this loop is split into different threads, then I suspect the randomness of the numbers is not assured.? What can I do about that? ? I can provide the loop code, but it?s fairly long, say 75-100 lines. ? If R/parallel is not feasible then a recompile with BLAS and pnmath appears to be the next best option.
R/parallel
4 messages · Scott Raynaud, Tal Galili, Brian Ripley
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20111208/a9206997/attachment.pl>
An embedded and charset-unspecified text was scrubbed... Name: not available URL: <https://stat.ethz.ch/pipermail/r-help/attachments/20111208/e3db96d7/attachment.pl>
On Thu, 8 Dec 2011, Scott Raynaud wrote:
Looks like this requires use of foreach and??lots of extra coding.?? R/parallel only requires that the loop be bracketed with the code to start the parallel processing-about 4-5 lines.?? Seems??a lot?? easier to me.?? If I were to go to the trouble of writing a lot of new code, it seems that recompiling with BLAS and pnmath would be a better option.
On Windows, that will be man-days of work unless you know the insides intimately (and no guarantees that it will eventually give a speed increase). There is no support for parallel BLAS nor OpenMP nor pthreads in the current R sources/binaries for Windows.
?? My main question is how to handle the random number generation when the child processes are spawned.?? That's a problem no matter what method I choose to create the threads.
A caveat: the R interpreter is not thread-safe: don't assume that you can run R code in parallel threads. I don't know what you mean by 'R/parallel'. However, R has a 'parallel' package, and its vignette discusses all this (including RNG). Adding parallel support using package 'parallel' is simple (and well-documented), not least as it comes ready-to-roll with R. If you meant the unfortunately named project at www.rparallel.org, a few comments: 1) It has not been updated in 3 years, and the pre-compiled Windows binaries are not going to work with recent R (like R >= 2.10.0). 2) It seems a lot less mature than the 'parallel' package and makes several restrictive assumptions. 3) Part of that lack of maturity is lack of documentation (including of the restrictive assumptions). 4) Inter-process communication seems to be by files. That is going to be slow, especially on Windows. Package 'parallel' uses sockets and pipes.
?? From: Tal Galili <tal.galili at gmail.com> To: Scott Raynaud <scott.raynaud at yahoo.com> Cc: "r-help at r-project.org" <r-help at r-project.org> Sent: Thursday, December 8, 2011 12:38 PM Subject: Re: [R] R/parallel Hi Scott, Why not use the doSMP package from REvolution? http://www.r-statistics.com/2010/04/parallel-multicore-processing-with-r-on-windows/ Tal ----------------Contact Details:------------------------------------------------------- Contact me: Tal.Galili at gmail.com |?? 972-52-7275845 Read me: www.talgalili.com (Hebrew) | www.biostatistics.co.il (Hebrew) | www.r-statistics.com (English) ---------------------------------------------------------------------------------------------- te: I want to take advantage?? of my multicore CPU to speed up a loop in a simulation program.?? I didn???t write the code,
but the iterations appear independent to me, at least in the sense that the results of one loop do not depend on previous ones.?? Right now I???m relegated to a Windows box that runs Windows 7.?? These appear to be the options: ?? Pnmath-appears to parallelize non-BLAS routine but requires a special build Fork-UNIX only Romp-looks like this hasn???t advanced past the developmental stage Multicore-use on Windows at your own risk R/parallel-seems like the best option if I don???t want to recompile. ?? Has anyone ever used R/parallel??? What kind of results did you have??? One difficulty with my simulation is that the loop includes code to generate random numbers.?? If this loop is split into different threads, then I suspect the randomness of the numbers is not assured.?? What can I do about that? ?? I can provide the loop code, but it???s fairly long, say 75-100 lines. ?? If R/parallel is not feasible then a recompile with BLAS and pnmath appears to be the next best option.
[[alternative HTML version deleted]]
Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595