Dear List,
the advent of multicore machines in the consumer segment makes me wonder
whether it would, at least in principle, be possible to divide a
computational task into more slave R processes running on the different
cores of the same processor, more or less in the way package SNOW would
do on a cluster. I am thinking of simple 'embarassingly parallel'
problems, just like inverting 1000 matrices, estimating 1000 models or
the like.
I have seen some talk here on making R multi-threaded and the like, but
this is much simpler. I am just a curious useR, so don't bother if you
don't have time, but maybe you can point me at some resource, or just
say "this is nonsense"...
Cheers
Giovanni
Giovanni Millo
Research Dept.,
Assicurazioni Generali SpA
Via Machiavelli 4,
34131 Trieste (Italy)
tel. +39 040 671184
fax +39 040 671160
Ai sensi del D.Lgs. 196/2003 si precisa che le informazioni ...{{dropped}}
simple parallel computing on single multicore machine
4 messages · Millo Giovanni, Uwe Ligges, Peter Dalgaard +1 more
Millo Giovanni wrote:
Dear List, the advent of multicore machines in the consumer segment makes me wonder whether it would, at least in principle, be possible to divide a computational task into more slave R processes running on the different cores of the same processor, more or less in the way package SNOW would do on a cluster. I am thinking of simple 'embarassingly parallel' problems, just like inverting 1000 matrices, estimating 1000 models or the like. I have seen some talk here on making R multi-threaded and the like, but this is much simpler. I am just a curious useR, so don't bother if you don't have time, but maybe you can point me at some resource, or just say "this is nonsense"...
Just use snow itself, for example. Or on a completely other level a tuned BLAS for perallel computations such as ATLAS. Uwe Ligges
Cheers
Giovanni
Giovanni Millo
Research Dept.,
Assicurazioni Generali SpA
Via Machiavelli 4,
34131 Trieste (Italy)
tel. +39 040 671184
fax +39 040 671160
Ai sensi del D.Lgs. 196/2003 si precisa che le informazioni ...{{dropped}}
______________________________________________ R-help at stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Millo Giovanni wrote:
Dear List, the advent of multicore machines in the consumer segment makes me wonder whether it would, at least in principle, be possible to divide a computational task into more slave R processes running on the different cores of the same processor, more or less in the way package SNOW would do on a cluster. I am thinking of simple 'embarassingly parallel' problems, just like inverting 1000 matrices, estimating 1000 models or the like. I have seen some talk here on making R multi-threaded and the like, but this is much simpler. I am just a curious useR, so don't bother if you don't have time, but maybe you can point me at some resource, or just say "this is nonsense"...
I don't think snow (or rather its underlying message-passing interface) cares whether its processes are on different physical machines. So this is easily doable. Of course you need to be aware that the processes are competing for resources like RAM and disc.
O__ ---- Peter Dalgaard ?ster Farimagsgade 5, Entr.B c/ /'_ --- Dept. of Biostatistics PO Box 2099, 1014 Cph. K (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 ~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk) FAX: (+45) 35327907
On Friday 01 December 2006 13:23, Millo Giovanni wrote:
Dear List, the advent of multicore machines in the consumer segment makes me wonder whether it would, at least in principle, be possible to divide a computational task into more slave R processes running on the different cores of the same processor, more or less in the way package SNOW would do on a cluster. I am thinking of simple 'embarassingly parallel' problems, just like inverting 1000 matrices, estimating 1000 models or the like. I have seen some talk here on making R multi-threaded and the like, but this is much simpler. I am just a curious useR, so don't bother if you don't have time, but maybe you can point me at some resource, or just say "this is nonsense"...
Dear Millo, I find the usage of papply (from the library with the same name), which itself uses Rmpi to be easy and ideal for those cases. The papply documentation shows clearly what you need to do to pass the required arguments to papply. And once you have your MPI universe up and running (with whichever number of slaves you specify) it just works. As well, I find debugging very simple: just start an MPI universe with only one node, which forces papply to run serially (non-parallel) so wrong arguments, missing libraries, etc, are easy to spot. Best, R.
Cheers
Giovanni
Giovanni Millo
Research Dept.,
Assicurazioni Generali SpA
Via Machiavelli 4,
34131 Trieste (Italy)
tel. +39 040 671184
fax +39 040 671160
Ai sensi del D.Lgs. 196/2003 si precisa che le informazioni ...{{dropped}}
______________________________________________ R-help at stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Ram?n D?az-Uriarte Centro Nacional de Investigaciones Oncol?gicas (CNIO) (Spanish National Cancer Center) Melchor Fern?ndez Almagro, 3 28029 Madrid (Spain) Fax: +-34-91-224-6972 Phone: +-34-91-224-6900 http://ligarto.org/rdiaz PGP KeyID: 0xE89B3462 (http://ligarto.org/rdiaz/0xE89B3462.asc) **NOTA DE CONFIDENCIALIDAD** Este correo electr?nico, y en s...{{dropped}}