mixing MP with MPI in R?
Mark Kimpel wrote:
I recently ran into a problem that was easier to solve using mutilcore compared with Rmpi because I had a large matrix that I was performing calculations on and it copying it for each process ate up my 12GB of memory. I now have access to 2 Linux machine, each a core i7 with 12GB of memory, and wonder how I might speed things up even faster by using some sort of combination of multicore and Rmpi/snow. I'm a novice at this, so perhaps the answer is obvious, but is it possible to spawn to multiple machines with non-shared memory but within each machine used shared memory? If my novice understanding is correct, the former uses openMPI and the latter openMP. If this is possible, a self-contained example would be appreciated, even if the calculations are so trivial as to not make the parallelization worthwhile in the example case
See: http://cran.r-project.org/web/packages/foreach/vignettes/nested.pdf This should work fine with doMPI/doSnow and doMC Regards, - Brian
Brian G. Peterson http://braverock.com/brian/ Ph: 773-459-4973 IM: bgpbraverock