dear members,
I am taking recourse to parallel computing to speed up my R code.
The mcparallel function(in parallel package),presumably, sends a single task (an R function, to be precise) to a single core. Is this right? Is that one and only one core? What if there are other cores lying idle?
As far as I am concerned, I am going to rent an AWS EC2 server with 48 cores. I have four functions to be parallelized. Can I send the thread of one function to 12 cores, so that all 48 cores are utilized? If so, how do I do that with mcparallel function in the parallel package?
Very many thanks for your time and effort...
yours sincerely....
AKSHAY M KULKARNI
parallel computing in r....
3 messages · akshay kulkarni, Jeff Newmiller, Juan Telleria Ruiz de Aguirre
2 days later
You cannot send one task to 12 processors... the price of parallelism is that you must break down your task into smaller tasks. Once you have number of tasks equal to our more than the number of available cores then the higher level functions such as parLapply or mclapply can shuffle tasks onto cores. If you have 48 cores and 4 tasks, only 4 of the cores will be used. You probably should stay away from directly calling mcparallel as it is a low level function. Read the vignette about the base "parallel" package [1]. You might also like to use the forEach contributed package to simplify your work, and should look at the CRAN High Performance Computing Task View [2].
[1] vignette("parallel") or http://stat.ethz.ch/R-manual/R-devel/library/parallel/doc/parallel.pdf
[2] https://cran.r-project.org/web/views/HighPerformanceComputing.html
On June 22, 2018 3:12:28 AM PDT, akshay kulkarni <akshay_e4 at hotmail.com> wrote:
dear members,
I am taking recourse to parallel computing to speed up my R code.
The mcparallel function(in parallel package),presumably, sends a single
task (an R function, to be precise) to a single core. Is this right? Is
that one and only one core? What if there are other cores lying idle?
As far as I am concerned, I am going to rent an AWS EC2 server with 48
cores. I have four functions to be parallelized. Can I send the thread
of one function to 12 cores, so that all 48 cores are utilized? If so,
how do I do that with mcparallel function in the parallel package?
Very many thanks for your time and effort...
yours sincerely....
AKSHAY M KULKARNI
[[alternative HTML version deleted]]
______________________________________________ R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Sent from my phone. Please excuse my brevity.
3 days later
Maybe this R-bloggers post could also help you: https://www.google.es/amp/s/www.r-bloggers.com/implementing-parallel-processing-in-r/amp/ https://www.google.es/amp/s/www.r-bloggers.com/a-guide-to-parallelism-in-r/amp/